2023-01-24T19:16:02.170 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T19:16:02.171 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655 --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655/orig.config.yaml 2023-01-24T19:16:02.216 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T19:16:02.286 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache} email: yweinstein@radhat.com first_in_suite: false job_id: '7135655' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore odsync write: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 2 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBTL2Y/PhS5EBGYHttlV9kpZvxPcjdAKTc/toHFeYydBHuV/RPQjTKn09plQ+Sq3Xgm2pdBKpe8sQFT5yQaEzhnKz9kNKCh8EiDQGj67Khw6lZ4U6RcwxdjXiAcfEaRBGatDaBHmrSryTK7259LXCx0lT266f03BoyVcuw3M00HzifXGL5bXdw8dLvZjVSSgkUvuUVZqSEmd9nIF6yBGaR4NPQhTjvnD9v6en9g1RBzMH1xgL4RIegNNBtkanG3pXo8oYKltKfS7GtyWSsgTcPvlNnzOA+RVijt2miZkUnVBN/b8HzHibKUMqHQSznKEI9/PJV2+CotOCGWUQRelO1JpM+uNmx9CWDH7rHgB/JwQDn4+kZ9aIC2N0aJ6lALIehYJoz55G0GEpTxBwMxeQxPYwyU1CFLkbYrLRyNHfofwJBtBt7loiXMxfqmUCj5glz5QLzLy+I3Nl2mIPbxGJGfGkbL4mBBuOJRGgVKmbfYPsAKtWq4FquYMhRsVq53sM= smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWJ9eq7e+26VETaENGgNVnn1rndNRtDKezG1HU7oVihEx6Y4cn7tNlK7WoqZxXFT3JQAUGoiU+K+Cj8SmtcJgmxXXY2zVKoqNgK2yPRfS2bTFaoAaZkCnnC/hs1gcxvZ70LYLtZikUk9F42s8c3G8G4JqVifdrVWPL5HS3yope/eJ9cTkh+7UUnM/yYo8wc9gBvPqU7443qLsuosDUW+HCF+leTR8lo83AvCZRPv+sakL2oP/zW4yq/qIr8swMSvQSVgAn3J7Ww9nBykyRIp2MozOBwSxr/Fb02YJK7WTSSuVVa7E3Q4R+G+IMziajjSGUeN0o/02FjwKWxaQVscUXooORlvC27U6lHxS9o8D6F5pCL0wdZAMjVGFTChl75SAFrVF2nYR9S+7TKpmFNa+cB5eTyAu+E7fcx9ndWV2Juig/vrhOXP2uYhK+TS5i5eJDOdjyOAugoHT88QP1lzXGLLAV/S5nKjY2c+9lxNqGm77Wv/SkccokUkjz6hbOQs= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache min_read_recency_for_promote 0 - sudo ceph osd pool set cache min_write_recency_for_promote 0 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T19:16:02.286 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T19:16:02.287 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T19:16:02.287 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T19:16:02.289 INFO:teuthology.task.internal:Checking packages... 2023-01-24T19:16:02.315 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T19:16:02.316 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T19:16:02.316 INFO:teuthology.packaging:ref: None 2023-01-24T19:16:02.317 INFO:teuthology.packaging:tag: None 2023-01-24T19:16:02.317 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T19:16:02.317 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:16:02.317 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T19:16:02.505 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T19:16:02.505 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T19:16:02.512 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T19:16:02.512 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T19:16:02.521 INFO:teuthology.task.internal:Saving configuration 2023-01-24T19:16:02.539 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T19:16:02.545 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T19:16:02.568 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 19:10:57.031749', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrneLaBNzpucrducbKVZcRPMmeMWmlvxXRF6BbhDrYs0SThyZrmvbt8zYzS7pwuQu6+JgveGIRYHHhMK4V0zfg='} 2023-01-24T19:16:02.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 19:10:57.030211', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ22lQ97/t9CKEB3QOe9xUpApepashdgytt+Y1Ete2xILWivn0F4PYRPevD8fXbAlToLHgHj37krLUTgPSLnXh4='} 2023-01-24T19:16:02.589 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T19:16:02.596 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T19:16:02.596 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T19:16:02.596 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T19:16:02.664 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5a877d0670>, signals=[15]) 2023-01-24T19:16:02.665 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T19:16:02.674 INFO:teuthology.task.internal:Opening connections... 2023-01-24T19:16:02.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:16:02.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:16:02.746 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:16:02.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:16:02.812 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T19:16:02.819 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-01-24T19:16:02.838 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-01-24T19:16:02.838 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-01-24T19:16:02.894 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T19:16:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T19:16:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-01-24T19:16:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-01-24T19:16:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.4" 2023-01-24T19:16:02.895 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-01-24T19:16:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T19:16:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-01-24T19:16:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T19:16:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T19:16:02.896 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T19:16:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T19:16:02.897 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:16:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T19:16:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T19:16:02.897 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T19:16:02.898 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T19:16:02.898 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-01-24T19:16:02.913 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-01-24T19:16:02.930 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-01-24T19:16:02.930 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-01-24T19:16:02.986 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T19:16:02.986 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T19:16:02.986 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2023-01-24T19:16:02.986 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2023-01-24T19:16:02.986 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.4" 2023-01-24T19:16:02.987 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-01-24T19:16:02.987 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T19:16:02.987 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-01-24T19:16:02.987 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T19:16:02.987 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T19:16:02.988 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T19:16:02.988 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T19:16:02.988 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:16:02.988 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T19:16:02.988 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T19:16:02.989 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T19:16:02.989 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T19:16:02.989 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-01-24T19:16:03.003 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T19:16:03.013 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T19:16:03.025 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T19:16:03.025 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-01-24T19:16:03.028 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2023-01-24T19:16:03.043 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T19:16:03.050 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T19:16:03.050 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-01-24T19:16:03.085 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2023-01-24T19:16:03.099 INFO:teuthology.orchestra.run.smithi062.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T19:16:03.101 INFO:teuthology.orchestra.run.smithi105.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T19:16:03.102 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T19:16:03.152 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T19:16:03.166 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T19:16:03.166 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T19:16:03.166 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T19:16:03.167 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T19:16:03.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:16:03.182 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-01-24T19:16:03.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:16:03.184 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-01-24T19:16:03.238 INFO:teuthology.orchestra.run.smithi105.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:03.238 INFO:teuthology.task.kernel:Running kernel on smithi105: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:03.238 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2023-01-24T19:16:03.240 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:03.241 INFO:teuthology.task.kernel:Running kernel on smithi062: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:03.241 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2023-01-24T19:16:04.653 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:04.654 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2023-01-24T19:16:04.730 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:04.731 INFO:teuthology.orchestra.run.smithi105.stdout:Unable to read consumer identity 2023-01-24T19:16:06.093 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-01-24T19:16:06.112 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-01-24T19:16:07.016 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-24T19:16:07.072 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-24T19:16:09.798 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 1.4 MB/s | 24 kB 00:00 2023-01-24T19:16:10.074 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 3.9 MB/s | 24 kB 00:00 2023-01-24T19:16:10.558 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T19:16:10.559 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T19:16:10.559 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T19:16:10.692 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:16:10.693 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:16:10.693 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:16:10.776 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T19:16:10.777 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T19:16:10.777 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T19:16:10.799 DEBUG:teuthology.orchestra.run.smithi105:> echo no | sudo yum reinstall kernel || true 2023-01-24T19:16:10.908 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:16:10.909 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:16:10.910 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:16:11.012 DEBUG:teuthology.orchestra.run.smithi062:> echo no | sudo yum reinstall kernel || true 2023-01-24T19:16:11.141 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:11.141 INFO:teuthology.orchestra.run.smithi105.stdout:Unable to read consumer identity 2023-01-24T19:16:11.380 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:11.381 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2023-01-24T19:16:11.431 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 07:16:09 PM UTC. 2023-01-24T19:16:11.574 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T19:16:11.575 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:11.575 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:11.626 INFO:teuthology.orchestra.run.smithi105.stderr:Error: No packages marked for reinstall. 2023-01-24T19:16:11.675 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum reinstall -y kernel || true 2023-01-24T19:16:11.678 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:01 ago on Tue 24 Jan 2023 07:16:10 PM UTC. 2023-01-24T19:16:11.806 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T19:16:11.807 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:11.807 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:11.831 INFO:teuthology.orchestra.run.smithi062.stderr:Error: No packages marked for reinstall. 2023-01-24T19:16:11.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum reinstall -y kernel || true 2023-01-24T19:16:12.019 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:12.019 INFO:teuthology.orchestra.run.smithi105.stdout:Unable to read consumer identity 2023-01-24T19:16:12.246 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:16:12.247 INFO:teuthology.orchestra.run.smithi062.stdout:Unable to read consumer identity 2023-01-24T19:16:12.296 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:03 ago on Tue 24 Jan 2023 07:16:09 PM UTC. 2023-01-24T19:16:12.443 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T19:16:12.443 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:12.443 INFO:teuthology.orchestra.run.smithi105.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:12.477 INFO:teuthology.orchestra.run.smithi105.stderr:Error: No packages marked for reinstall. 2023-01-24T19:16:12.526 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel --last | head -n 1 2023-01-24T19:16:12.541 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 07:16:10 PM UTC. 2023-01-24T19:16:12.650 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T19:16:12.652 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:12.652 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T19:16:12.652 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T19:16:12.653 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T19:16:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2023-01-24T19:16:12.700 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T19:16:12.700 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:12.700 INFO:teuthology.orchestra.run.smithi062.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T19:16:12.701 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:Error: No packages marked for reinstall. 2023-01-24T19:16:12.767 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel --last | head -n 1 2023-01-24T19:16:12.895 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T19:16:12.897 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T19:16:12.897 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T19:16:12.897 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T19:16:12.898 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T19:16:12.942 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2023-01-24T19:16:12.956 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:12.956 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T19:16:12.963 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T19:16:12.964 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T19:16:12.966 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T19:16:12.980 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T19:16:12.987 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T19:16:12.998 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T19:16:12.998 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T19:16:13.021 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T19:16:13.050 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T19:16:13.058 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T19:16:13.058 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T19:16:13.081 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T19:16:13.108 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:16:13.115 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:16:13.118 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:16:13.124 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:16:13.126 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T19:16:13.133 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T19:16:13.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T19:16:13.162 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T19:16:13.189 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T19:16:13.197 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T19:16:13.198 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T19:16:13.227 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T19:16:13.244 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:16:13.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:16:13.367 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:16:13.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:16:13.487 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:16:13.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T19:16:13.551 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:16:13.596 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:16:13.632 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:16:13.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:16:13.750 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:16:13.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T19:16:13.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-01-24T19:16:13.817 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2023-01-24T19:16:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T19:16:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T19:16:14.281 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T19:16:14.290 INFO:teuthology.task.internal:Starting timer... 2023-01-24T19:16:14.290 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T19:16:14.300 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T19:16:14.308 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T19:16:14.309 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-01-24T19:16:14.370 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: [ OK ] 2023-01-24T19:16:14.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2023-01-24T19:16:14.426 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: [ OK ] 2023-01-24T19:16:14.428 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T19:16:14.428 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-01-24T19:16:14.449 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-01-24T19:16:14.449 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2023-01-24T19:16:14.473 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2023-01-24T19:16:14.473 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2023-01-24T19:16:14.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T19:16:14.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:16:14.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T19:16:14.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:16:14.539 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T19:16:14.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-01-24T19:16:14.582 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2023-01-24T19:16:14.606 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T19:16:14.616 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-24T19:16:14.621 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-24T19:16:14.746 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T19:16:14.761 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T19:16:14.762 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T19:22:51.156 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2023-01-24T19:22:51.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-01-24T19:22:51.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:22:51.227 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-01-24T19:22:51.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-01-24T19:22:51.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:22:51.362 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-01-24T19:22:51.428 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T19:22:51.441 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T19:22:51.442 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T19:22:51.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T19:22:51.445 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T19:22:51.445 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T19:22:51.472 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T19:22:51.486 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T19:22:51.499 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T19:22:51.513 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T19:22:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-01-24T19:22:51.533 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-01-24T19:22:51.547 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T19:22:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2023-01-24T19:22:51.561 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T19:22:51.561 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2023-01-24T19:22:51.574 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T19:22:51.587 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T19:22:51.624 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-01-24T19:22:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:210 Number of sources = 4 2023-01-24T19:22:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T19:22:51.627 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-01-24T19:22:51.628 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.628 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.628 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.628 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.647 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2023-01-24T19:22:51.650 INFO:teuthology.orchestra.run.smithi105.stdout:210 Number of sources = 4 2023-01-24T19:22:51.650 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T19:22:51.651 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2023-01-24T19:22:51.651 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.651 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.651 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.652 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:22:51.652 INFO:teuthology.run_tasks:Running task install... 2023-01-24T19:22:51.669 DEBUG:teuthology.task.install:project ceph 2023-01-24T19:22:51.669 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T19:22:51.670 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T19:22:51.670 INFO:teuthology.task.install:Using flavor: default 2023-01-24T19:22:51.676 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T19:22:51.676 INFO:teuthology.task.install:extra packages: [] 2023-01-24T19:22:51.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-24T19:22:51.715 INFO:teuthology.orchestra.run.smithi062.stdout:RedHatEnterprise 2023-01-24T19:22:51.717 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:22:51.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T19:22:51.718 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T19:22:51.718 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-24T19:22:51.755 INFO:teuthology.orchestra.run.smithi105.stdout:RedHatEnterprise 2023-01-24T19:22:51.756 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:22:51.757 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T19:22:51.757 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T19:22:51.796 INFO:teuthology.orchestra.run.smithi062.stdout:RedHatEnterprise 2023-01-24T19:22:51.797 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:22:51.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:22:51.838 INFO:teuthology.orchestra.run.smithi105.stdout:RedHatEnterprise 2023-01-24T19:22:51.839 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:22:51.839 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:22:51.970 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T19:22:51.971 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T19:22:52.048 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T19:22:52.048 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T19:22:52.193 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T19:22:52.193 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:22:52.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T19:22:52.219 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T19:22:52.220 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T19:22:52.240 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T19:22:52.240 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:22:52.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T19:22:52.265 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T19:22:52.265 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T19:22:52.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T19:22:52.329 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T19:22:52.361 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T19:22:52.402 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T19:22:52.426 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-01-24T19:22:52.428 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-01-24T19:22:52.466 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2023-01-24T19:22:52.467 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2023-01-24T19:22:52.835 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:22:52.851 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:22:53.157 INFO:teuthology.orchestra.run.smithi062.stdout:44 files removed 2023-01-24T19:22:53.171 INFO:teuthology.orchestra.run.smithi105.stdout:44 files removed 2023-01-24T19:22:53.208 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw 2023-01-24T19:22:53.219 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw 2023-01-24T19:22:53.564 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:22:53.574 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:22:54.417 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 188 kB/s | 78 kB 00:00 2023-01-24T19:22:54.442 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 186 kB/s | 78 kB 00:00 2023-01-24T19:22:54.804 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 27 kB/s | 9.2 kB 00:00 2023-01-24T19:22:54.833 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 26 kB/s | 9.2 kB 00:00 2023-01-24T19:22:55.180 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-01-24T19:22:55.213 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-01-24T19:22:55.283 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 46 kB/s | 3.5 kB 00:00 2023-01-24T19:22:55.321 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 45 kB/s | 3.5 kB 00:00 2023-01-24T19:22:56.679 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 13 MB 00:01 2023-01-24T19:22:56.833 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 13 MB 00:01 2023-01-24T19:22:59.552 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-01-24T19:22:59.739 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-01-24T19:22:59.895 INFO:teuthology.orchestra.run.smithi105.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 MB/s | 33 MB 00:00 2023-01-24T19:23:00.133 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 105 MB/s | 33 MB 00:00 2023-01-24T19:23:04.992 INFO:teuthology.orchestra.run.smithi105.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 130 MB/s | 38 MB 00:00 2023-01-24T19:23:05.538 INFO:teuthology.orchestra.run.smithi062.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 114 MB/s | 38 MB 00:00 2023-01-24T19:23:13.754 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:23:13.756 INFO:teuthology.orchestra.run.smithi105.stdout:=================================================================================================== 2023-01-24T19:23:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:23:13.757 INFO:teuthology.orchestra.run.smithi105.stdout:=================================================================================================== 2023-01-24T19:23:13.757 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:23:13.757 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:23:13.757 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T19:23:13.758 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T19:23:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T19:23:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T19:23:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T19:23:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T19:23:13.759 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T19:23:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T19:23:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T19:23:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T19:23:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T19:23:13.760 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T19:23:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T19:23:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T19:23:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T19:23:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T19:23:13.761 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T19:23:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T19:23:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T19:23:13.762 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:23:13.762 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:23:13.762 INFO:teuthology.orchestra.run.smithi105.stdout:=================================================================================================== 2023-01-24T19:23:13.763 INFO:teuthology.orchestra.run.smithi105.stdout:Install 21 Packages 2023-01-24T19:23:13.763 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 2 Packages 2023-01-24T19:23:13.763 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:23:13.764 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 63 M 2023-01-24T19:23:13.764 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:23:14.850 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:23:14.853 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================== 2023-01-24T19:23:14.853 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:23:14.853 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================== 2023-01-24T19:23:14.853 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:23:14.854 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:23:14.854 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-01-24T19:23:14.854 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:23:14.854 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T19:23:14.855 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:23:14.855 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T19:23:14.855 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T19:23:14.855 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T19:23:14.855 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T19:23:14.856 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T19:23:14.856 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T19:23:14.856 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T19:23:14.857 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T19:23:14.857 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T19:23:14.857 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T19:23:14.857 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T19:23:14.858 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T19:23:14.859 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T19:23:14.859 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T19:23:14.859 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T19:23:14.859 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:14.859 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:23:14.860 INFO:teuthology.orchestra.run.smithi062.stdout:=================================================================================================== 2023-01-24T19:23:14.860 INFO:teuthology.orchestra.run.smithi062.stdout:Install 21 Packages 2023-01-24T19:23:14.860 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 2 Packages 2023-01-24T19:23:14.860 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:14.861 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 63 M 2023-01-24T19:23:14.861 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:23:16.351 INFO:teuthology.orchestra.run.smithi062.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 3.4 MB/s | 5.1 MB 00:01 2023-01-24T19:23:16.383 INFO:teuthology.orchestra.run.smithi062.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 758 kB/s | 24 kB 00:00 2023-01-24T19:23:16.509 INFO:teuthology.orchestra.run.smithi062.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 6.2 MB/s | 793 kB 00:00 2023-01-24T19:23:16.596 INFO:teuthology.orchestra.run.smithi062.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 5.7 MB/s | 506 kB 00:00 2023-01-24T19:23:17.344 INFO:teuthology.orchestra.run.smithi062.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 7.4 MB/s | 5.5 MB 00:00 2023-01-24T19:23:17.409 INFO:teuthology.orchestra.run.smithi062.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 697 kB/s | 45 kB 00:00 2023-01-24T19:23:17.478 INFO:teuthology.orchestra.run.smithi062.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.5 MB/s | 108 kB 00:00 2023-01-24T19:23:17.551 INFO:teuthology.orchestra.run.smithi062.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 2.6 MB/s | 190 kB 00:00 2023-01-24T19:23:17.696 INFO:teuthology.orchestra.run.smithi062.stdout:(9/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8.x 5.9 MB/s | 17 MB 00:02 2023-01-24T19:23:17.699 INFO:teuthology.orchestra.run.smithi062.stdout:(10/23): python3-rados-17.2.5-727.g49f8fb05.el8 2.4 MB/s | 370 kB 00:00 2023-01-24T19:23:17.785 INFO:teuthology.orchestra.run.smithi062.stdout:(11/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 3.9 MB/s | 357 kB 00:00 2023-01-24T19:23:17.789 INFO:teuthology.orchestra.run.smithi062.stdout:(12/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 1.2 MB/s | 107 kB 00:00 2023-01-24T19:23:18.088 INFO:teuthology.orchestra.run.smithi105.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 1.2 MB/s | 5.1 MB 00:04 2023-01-24T19:23:18.119 INFO:teuthology.orchestra.run.smithi105.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 799 kB/s | 24 kB 00:00 2023-01-24T19:23:18.169 INFO:teuthology.orchestra.run.smithi062.stdout:(13/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 7.0 MB/s | 23 MB 00:03 2023-01-24T19:23:18.170 INFO:teuthology.orchestra.run.smithi062.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 155 kB/s | 59 kB 00:00 2023-01-24T19:23:18.222 INFO:teuthology.orchestra.run.smithi062.stdout:(15/23): gperftools-libs-2.7-9.el8.x86_64.rpm 701 kB/s | 306 kB 00:00 2023-01-24T19:23:18.265 INFO:teuthology.orchestra.run.smithi062.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-01-24T19:23:18.282 INFO:teuthology.orchestra.run.smithi062.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-01-24T19:23:18.300 INFO:teuthology.orchestra.run.smithi062.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 576 kB/s | 75 kB 00:00 2023-01-24T19:23:18.302 INFO:teuthology.orchestra.run.smithi062.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 17 MB/s | 353 kB 00:00 2023-01-24T19:23:18.342 INFO:teuthology.orchestra.run.smithi062.stdout:(20/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-01-24T19:23:18.375 INFO:teuthology.orchestra.run.smithi105.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 3.0 MB/s | 793 kB 00:00 2023-01-24T19:23:18.647 INFO:teuthology.orchestra.run.smithi062.stdout:(21/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 12 MB/s | 3.6 MB 00:00 2023-01-24T19:23:18.667 INFO:teuthology.orchestra.run.smithi062.stdout:(22/23): librados2-17.2.5-727.g49f8fb05.el8.x86 9.5 MB/s | 3.5 MB 00:00 2023-01-24T19:23:18.674 INFO:teuthology.orchestra.run.smithi105.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 1.7 MB/s | 506 kB 00:00 2023-01-24T19:23:19.094 INFO:teuthology.orchestra.run.smithi062.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-01-24T19:23:19.097 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:23:19.098 INFO:teuthology.orchestra.run.smithi062.stdout:Total 15 MB/s | 63 MB 00:04 2023-01-24T19:23:19.103 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:23:19.153 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:23:19.154 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:23:19.898 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:23:19.902 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:23:20.570 INFO:teuthology.orchestra.run.smithi105.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 2.9 MB/s | 5.5 MB 00:01 2023-01-24T19:23:20.605 INFO:teuthology.orchestra.run.smithi105.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 1.3 MB/s | 45 kB 00:00 2023-01-24T19:23:20.641 INFO:teuthology.orchestra.run.smithi105.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 2.9 MB/s | 108 kB 00:00 2023-01-24T19:23:20.728 INFO:teuthology.orchestra.run.smithi105.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 2.2 MB/s | 190 kB 00:00 2023-01-24T19:23:20.848 INFO:teuthology.orchestra.run.smithi105.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 3.0 MB/s | 370 kB 00:00 2023-01-24T19:23:20.956 INFO:teuthology.orchestra.run.smithi105.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 3.3 MB/s | 357 kB 00:00 2023-01-24T19:23:20.984 INFO:teuthology.orchestra.run.smithi105.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 3.7 MB/s | 107 kB 00:00 2023-01-24T19:23:21.209 INFO:teuthology.orchestra.run.smithi105.stdout:(12/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 2.2 MB/s | 17 MB 00:07 2023-01-24T19:23:21.254 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:23:21.270 INFO:teuthology.orchestra.run.smithi105.stdout:(13/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2023-01-24T19:23:21.337 INFO:teuthology.orchestra.run.smithi105.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-01-24T19:23:21.341 INFO:teuthology.orchestra.run.smithi105.stdout:(15/23): liboath-2.6.2-3.el8.x86_64.rpm 445 kB/s | 59 kB 00:00 2023-01-24T19:23:21.370 INFO:teuthology.orchestra.run.smithi105.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 1.5 MB/s | 44 kB 00:00 2023-01-24T19:23:21.387 INFO:teuthology.orchestra.run.smithi105.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-01-24T19:23:21.408 INFO:teuthology.orchestra.run.smithi105.stdout:(18/23): librdkafka-0.11.4-1.el8.x86_64.rpm 16 MB/s | 353 kB 00:00 2023-01-24T19:23:21.429 INFO:teuthology.orchestra.run.smithi105.stdout:(19/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.2 MB/s | 47 kB 00:00 2023-01-24T19:23:21.450 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T19:23:21.864 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T19:23:21.932 INFO:teuthology.orchestra.run.smithi105.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 2.9 MB/s | 1.7 MB 00:00 2023-01-24T19:23:22.075 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:22.176 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:22.326 INFO:teuthology.orchestra.run.smithi105.stdout:(21/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 2.7 MB/s | 23 MB 00:08 2023-01-24T19:23:22.357 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:23:22.424 INFO:teuthology.orchestra.run.smithi105.stdout:(22/23): librados2-17.2.5-727.g49f8fb05.el8.x86 3.5 MB/s | 3.5 MB 00:00 2023-01-24T19:23:22.638 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T19:23:22.850 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:22.968 INFO:teuthology.orchestra.run.smithi105.stdout:(23/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 3.5 MB/s | 3.6 MB 00:01 2023-01-24T19:23:22.971 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:23:22.972 INFO:teuthology.orchestra.run.smithi105.stdout:Total 6.8 MB/s | 63 MB 00:09 2023-01-24T19:23:22.977 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:23:23.023 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:23:23.023 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:23:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:23.484 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:23.564 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:23.750 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-01-24T19:23:23.765 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:23:23.770 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:23:23.934 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T19:23:24.159 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T19:23:24.878 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T19:23:24.957 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:23:25.025 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:25.112 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:25.178 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T19:23:25.245 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T19:23:25.454 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:23:25.589 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T19:23:25.662 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:23:25.828 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:25.909 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:26.090 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T19:23:26.126 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:23:26.258 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:23:26.318 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:23:26.352 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T19:23:26.527 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T19:23:26.536 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:26.712 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T19:23:26.936 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:26.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T19:23:27.144 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:27.234 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:27.267 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T19:23:27.435 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/25 2023-01-24T19:23:27.661 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T19:23:27.884 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/25 2023-01-24T19:23:28.560 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T19:23:28.744 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:28.814 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:29.243 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T19:23:29.415 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:23:29.647 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:23:29.882 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T19:23:30.028 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:23:30.139 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:23:30.330 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T19:23:30.548 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T19:23:30.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T19:23:31.053 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T19:23:32.341 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:32.591 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:33.199 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:33.500 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:23:33.589 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:23:33.716 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:35.937 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:36.194 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:36.781 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:23:37.085 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:23:37.162 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:23:37.293 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:52.148 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:52.149 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-01-24T19:23:52.149 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-01-24T19:23:52.149 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-01-24T19:23:52.149 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-01-24T19:23:52.149 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-01-24T19:23:52.150 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-01-24T19:23:52.150 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-01-24T19:23:52.150 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-01-24T19:23:52.150 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-01-24T19:23:52.150 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:53.981 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:23:54.057 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:23:54.057 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:23:54.257 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:23:54.257 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:54.790 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:54.790 INFO:teuthology.orchestra.run.smithi062.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T19:23:54.790 INFO:teuthology.orchestra.run.smithi062.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T19:23:54.791 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:55.048 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:55.048 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2023-01-24T19:23:55.048 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2023-01-24T19:23:55.049 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-01-24T19:23:55.049 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /run 2023-01-24T19:23:55.049 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2023-01-24T19:23:55.049 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2023-01-24T19:23:55.049 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2023-01-24T19:23:55.050 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2023-01-24T19:23:55.050 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-01-24T19:23:55.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:23:55.087 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T19:23:55.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:55.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:23:55.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:23:55.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:55.088 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:55.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T19:23:55.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T19:23:55.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T19:23:55.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:55.089 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T19:23:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:23:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:23:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T19:23:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T19:23:55.090 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T19:23:55.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T19:23:55.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T19:23:55.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-24T19:23:55.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-01-24T19:23:55.091 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-01-24T19:23:55.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:55.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T19:23:55.092 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T19:23:56.645 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:56.646 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:51 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:23:56.766 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:23:56.870 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:23:56.870 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:23:57.059 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:23:57.059 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:57.609 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:57.610 INFO:teuthology.orchestra.run.smithi105.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T19:23:57.610 INFO:teuthology.orchestra.run.smithi105.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T19:23:57.610 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:23:57.913 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T19:23:57.913 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:23:57.913 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:23:57.914 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:23:57.914 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:23:57.914 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:23:57.914 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T19:23:57.915 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T19:23:57.915 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T19:23:57.915 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:23:57.915 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T19:23:57.915 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:23:57.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:23:57.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T19:23:57.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T19:23:57.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T19:23:57.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T19:23:57.917 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T19:23:57.917 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-24T19:23:57.917 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/25 2023-01-24T19:23:57.917 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-01-24T19:23:57.917 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:23:57.918 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T19:23:57.918 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T19:23:58.849 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:23:58.972 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:58.973 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-01-24T19:23:58.973 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.973 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.973 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:23:58.974 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.974 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.974 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.974 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.974 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T19:23:58.975 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.975 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T19:23:58.975 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T19:23:58.975 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.975 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T19:23:58.976 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.976 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T19:23:58.976 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T19:23:58.976 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.976 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.977 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.977 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T19:23:58.977 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.977 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.977 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:23:58.978 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T19:23:58.978 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:23:58.978 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:23:59.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-test 2023-01-24T19:23:59.419 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:23:59.490 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:23:59.502 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:00.423 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:55 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:01.630 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:24:01.738 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:01.739 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2023-01-24T19:24:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.739 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.739 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:24:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.740 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T19:24:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T19:24:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T19:24:01.741 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T19:24:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.742 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T19:24:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T19:24:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.743 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T19:24:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:01.744 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T19:24:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:01.745 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:01.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-test 2023-01-24T19:24:02.234 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:02.288 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:24:02.289 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================== 2023-01-24T19:24:02.289 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:24:02.289 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================== 2023-01-24T19:24:02.290 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:24:02.290 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T19:24:02.290 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:24:02.290 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T19:24:02.290 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T19:24:02.291 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T19:24:02.291 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T19:24:02.291 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T19:24:02.291 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:02.291 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:24:02.292 INFO:teuthology.orchestra.run.smithi062.stdout:=========================================================================================== 2023-01-24T19:24:02.292 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2023-01-24T19:24:02.292 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:02.293 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 53 M 2023-01-24T19:24:02.293 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 233 M 2023-01-24T19:24:02.293 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:24:02.295 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-01-24T19:24:02.344 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 3.3 MB/s | 161 kB 00:00 2023-01-24T19:24:02.358 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 13 MB/s | 187 kB 00:00 2023-01-24T19:24:02.372 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 21 MB/s | 302 kB 00:00 2023-01-24T19:24:02.538 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 705 kB/s | 174 kB 00:00 2023-01-24T19:24:03.160 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:59 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:05.021 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:24:05.023 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2023-01-24T19:24:05.023 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:24:05.024 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2023-01-24T19:24:05.024 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:24:05.024 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T19:24:05.024 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:24:05.024 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T19:24:05.025 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T19:24:05.025 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T19:24:05.025 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T19:24:05.025 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T19:24:05.025 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:05.026 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:24:05.026 INFO:teuthology.orchestra.run.smithi105.stdout:=========================================================================================== 2023-01-24T19:24:05.026 INFO:teuthology.orchestra.run.smithi105.stdout:Install 6 Packages 2023-01-24T19:24:05.026 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:05.026 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 53 M 2023-01-24T19:24:05.027 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 233 M 2023-01-24T19:24:05.027 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:24:05.027 INFO:teuthology.orchestra.run.smithi105.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 22 MB/s | 70 kB 00:00 2023-01-24T19:24:05.034 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 19 MB/s | 52 MB 00:02 2023-01-24T19:24:05.038 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:05.038 INFO:teuthology.orchestra.run.smithi062.stdout:Total 19 MB/s | 53 MB 00:02 2023-01-24T19:24:05.042 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:24:05.064 INFO:teuthology.orchestra.run.smithi105.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 4.4 MB/s | 161 kB 00:00 2023-01-24T19:24:05.068 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:24:05.069 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:24:05.078 INFO:teuthology.orchestra.run.smithi105.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 13 MB/s | 187 kB 00:00 2023-01-24T19:24:05.092 INFO:teuthology.orchestra.run.smithi105.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 20 MB/s | 302 kB 00:00 2023-01-24T19:24:05.234 INFO:teuthology.orchestra.run.smithi105.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 831 kB/s | 174 kB 00:00 2023-01-24T19:24:05.440 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:24:05.445 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:24:05.999 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:24:06.204 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T19:24:06.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T19:24:06.483 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T19:24:06.662 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T19:24:06.844 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T19:24:07.003 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:24:07.732 INFO:teuthology.orchestra.run.smithi105.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 19 MB/s | 52 MB 00:02 2023-01-24T19:24:07.736 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:07.736 INFO:teuthology.orchestra.run.smithi105.stdout:Total 20 MB/s | 53 MB 00:02 2023-01-24T19:24:07.740 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:24:07.763 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:24:07.763 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:24:08.116 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:24:08.121 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:24:08.654 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:24:08.856 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T19:24:09.047 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T19:24:09.161 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T19:24:09.304 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T19:24:09.638 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T19:24:09.813 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:24:12.303 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:24:12.526 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:24:12.798 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:24:12.798 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T19:24:12.798 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T19:24:12.799 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T19:24:12.799 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T19:24:12.799 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T19:24:14.054 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T19:24:14.055 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:09 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:14.843 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:24:15.120 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:24:15.373 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:24:15.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T19:24:15.374 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T19:24:15.374 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T19:24:15.374 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T19:24:15.374 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T19:24:16.040 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:24:16.149 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:16.149 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:24:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.5-12.el8.x86_64 2023-01-24T19:24:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:16.150 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T19:24:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T19:24:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T19:24:16.151 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:16.151 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:24:16.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph 2023-01-24T19:24:16.486 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T19:24:16.487 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:12 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:16.777 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:17.651 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:12 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:18.476 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:24:18.572 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:18.574 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:24:18.574 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:18.574 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.5-12.el8.x86_64 2023-01-24T19:24:18.575 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:18.575 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T19:24:18.575 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T19:24:18.575 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T19:24:18.575 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:18.576 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:18.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph 2023-01-24T19:24:19.029 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:19.548 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:24:19.551 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================== 2023-01-24T19:24:19.551 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:24:19.551 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================== 2023-01-24T19:24:19.552 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:24:19.552 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T19:24:19.552 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:24:19.552 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T19:24:19.552 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T19:24:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T19:24:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T19:24:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:24:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T19:24:19.553 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T19:24:19.554 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T19:24:19.555 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T19:24:19.555 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T19:24:19.555 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T19:24:19.555 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T19:24:19.555 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T19:24:19.556 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T19:24:19.556 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T19:24:19.556 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T19:24:19.556 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T19:24:19.556 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:19.557 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:24:19.558 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================== 2023-01-24T19:24:19.558 INFO:teuthology.orchestra.run.smithi062.stdout:Install 28 Packages 2023-01-24T19:24:19.558 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:19.559 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 27 M 2023-01-24T19:24:19.559 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 105 M 2023-01-24T19:24:19.559 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:24:19.804 INFO:teuthology.orchestra.run.smithi062.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 26 kB/s | 6.4 kB 00:00 2023-01-24T19:24:19.880 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:15 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:21.232 INFO:teuthology.orchestra.run.smithi062.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 954 kB/s | 1.6 MB 00:01 2023-01-24T19:24:21.425 INFO:teuthology.orchestra.run.smithi062.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 1.2 MB/s | 2.3 MB 00:01 2023-01-24T19:24:21.474 INFO:teuthology.orchestra.run.smithi062.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 4.7 MB/s | 233 kB 00:00 2023-01-24T19:24:21.584 INFO:teuthology.orchestra.run.smithi062.stdout:(5/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 399 kB/s | 44 kB 00:00 2023-01-24T19:24:21.632 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:24:21.634 INFO:teuthology.orchestra.run.smithi062.stdout:(6/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 2.1 MB/s | 3.9 MB 00:01 2023-01-24T19:24:21.635 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================== 2023-01-24T19:24:21.635 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:24:21.635 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================== 2023-01-24T19:24:21.636 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:24:21.636 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T19:24:21.636 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:24:21.636 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T19:24:21.637 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T19:24:21.637 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T19:24:21.637 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T19:24:21.637 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:24:21.637 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T19:24:21.638 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T19:24:21.638 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T19:24:21.638 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T19:24:21.638 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T19:24:21.638 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T19:24:21.639 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T19:24:21.639 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T19:24:21.639 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T19:24:21.639 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T19:24:21.639 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T19:24:21.640 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T19:24:21.640 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T19:24:21.640 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T19:24:21.640 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T19:24:21.640 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T19:24:21.641 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T19:24:21.641 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T19:24:21.641 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T19:24:21.641 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T19:24:21.641 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T19:24:21.642 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T19:24:21.642 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:21.642 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:24:21.642 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================== 2023-01-24T19:24:21.642 INFO:teuthology.orchestra.run.smithi105.stdout:Install 28 Packages 2023-01-24T19:24:21.643 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:21.643 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 27 M 2023-01-24T19:24:21.643 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 105 M 2023-01-24T19:24:21.644 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:24:21.666 INFO:teuthology.orchestra.run.smithi062.stdout:(7/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.2 MB/s | 185 kB 00:00 2023-01-24T19:24:21.739 INFO:teuthology.orchestra.run.smithi105.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 64 kB/s | 6.4 kB 00:00 2023-01-24T19:24:21.803 INFO:teuthology.orchestra.run.smithi062.stdout:(8/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.0 MB/s | 173 kB 00:00 2023-01-24T19:24:21.805 INFO:teuthology.orchestra.run.smithi062.stdout:(9/28): python3-cherrypy-18.4.0-1.el8.noarch.rp 2.7 MB/s | 384 kB 00:00 2023-01-24T19:24:21.832 INFO:teuthology.orchestra.run.smithi062.stdout:(10/28): python3-jaraco-6.2-6.el8.noarch.rpm 395 kB/s | 11 kB 00:00 2023-01-24T19:24:21.835 INFO:teuthology.orchestra.run.smithi062.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 579 kB/s | 18 kB 00:00 2023-01-24T19:24:21.867 INFO:teuthology.orchestra.run.smithi062.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.4 MB/s | 49 kB 00:00 2023-01-24T19:24:21.872 INFO:teuthology.orchestra.run.smithi062.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 1.6 MB/s | 59 kB 00:00 2023-01-24T19:24:21.908 INFO:teuthology.orchestra.run.smithi062.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 430 kB/s | 16 kB 00:00 2023-01-24T19:24:21.937 INFO:teuthology.orchestra.run.smithi062.stdout:(15/28): python3-simplegeneric-0.8.1-17.el8.noa 662 kB/s | 19 kB 00:00 2023-01-24T19:24:21.967 INFO:teuthology.orchestra.run.smithi062.stdout:(16/28): python3-singledispatch-3.4.0.3-18.el8. 804 kB/s | 24 kB 00:00 2023-01-24T19:24:21.984 INFO:teuthology.orchestra.run.smithi062.stdout:(17/28): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-01-24T19:24:21.995 INFO:teuthology.orchestra.run.smithi062.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T19:24:21.998 INFO:teuthology.orchestra.run.smithi105.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 4.4 MB/s | 1.6 MB 00:00 2023-01-24T19:24:22.013 INFO:teuthology.orchestra.run.smithi062.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 958 kB/s | 27 kB 00:00 2023-01-24T19:24:22.037 INFO:teuthology.orchestra.run.smithi105.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 5.8 MB/s | 2.3 MB 00:00 2023-01-24T19:24:22.089 INFO:teuthology.orchestra.run.smithi105.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 4.4 MB/s | 233 kB 00:00 2023-01-24T19:24:22.099 INFO:teuthology.orchestra.run.smithi062.stdout:(20/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.8 MB/s | 251 kB 00:00 2023-01-24T19:24:22.109 INFO:teuthology.orchestra.run.smithi062.stdout:(21/28): python3-waitress-1.2.1-2.el8.1.noarch. 2.1 MB/s | 241 kB 00:00 2023-01-24T19:24:22.131 INFO:teuthology.orchestra.run.smithi062.stdout:(22/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.7 MB/s | 85 kB 00:00 2023-01-24T19:24:22.138 INFO:teuthology.orchestra.run.smithi062.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 811 kB/s | 23 kB 00:00 2023-01-24T19:24:22.173 INFO:teuthology.orchestra.run.smithi062.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 11 MB/s | 457 kB 00:00 2023-01-24T19:24:22.175 INFO:teuthology.orchestra.run.smithi062.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.0 MB/s | 39 kB 00:00 2023-01-24T19:24:22.189 INFO:teuthology.orchestra.run.smithi062.stdout:(26/28): python3-mako-1.0.6-13.el8.noarch.rpm 9.7 MB/s | 157 kB 00:00 2023-01-24T19:24:22.190 INFO:teuthology.orchestra.run.smithi062.stdout:(27/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.5 MB/s | 54 kB 00:00 2023-01-24T19:24:22.496 INFO:teuthology.orchestra.run.smithi105.stdout:(5/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 5.2 MB/s | 3.9 MB 00:00 2023-01-24T19:24:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:(28/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 9.3 MB/s | 17 MB 00:01 2023-01-24T19:24:23.024 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:23.025 INFO:teuthology.orchestra.run.smithi062.stdout:Total 7.8 MB/s | 27 MB 00:03 2023-01-24T19:24:23.030 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:24:23.034 INFO:teuthology.orchestra.run.smithi105.stdout:(6/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 46 kB/s | 44 kB 00:00 2023-01-24T19:24:23.074 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:24:23.074 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:24:23.180 INFO:teuthology.orchestra.run.smithi105.stdout:(7/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_6 14 MB/s | 17 MB 00:01 2023-01-24T19:24:23.180 INFO:teuthology.orchestra.run.smithi105.stdout:(8/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 270 kB/s | 185 kB 00:00 2023-01-24T19:24:23.181 INFO:teuthology.orchestra.run.smithi105.stdout:(9/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.1 MB/s | 173 kB 00:00 2023-01-24T19:24:23.208 INFO:teuthology.orchestra.run.smithi105.stdout:(10/28): python3-jaraco-6.2-6.el8.noarch.rpm 407 kB/s | 11 kB 00:00 2023-01-24T19:24:23.209 INFO:teuthology.orchestra.run.smithi105.stdout:(11/28): python3-jaraco-functools-2.0-4.el8.noa 631 kB/s | 18 kB 00:00 2023-01-24T19:24:23.236 INFO:teuthology.orchestra.run.smithi105.stdout:(12/28): python3-logutils-0.3.5-11.el8.noarch.r 1.7 MB/s | 49 kB 00:00 2023-01-24T19:24:23.238 INFO:teuthology.orchestra.run.smithi105.stdout:(13/28): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2023-01-24T19:24:23.269 INFO:teuthology.orchestra.run.smithi105.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 514 kB/s | 16 kB 00:00 2023-01-24T19:24:23.297 INFO:teuthology.orchestra.run.smithi105.stdout:(15/28): python3-simplegeneric-0.8.1-17.el8.noa 654 kB/s | 19 kB 00:00 2023-01-24T19:24:23.330 INFO:teuthology.orchestra.run.smithi105.stdout:(16/28): python3-singledispatch-3.4.0.3-18.el8. 750 kB/s | 24 kB 00:00 2023-01-24T19:24:23.354 INFO:teuthology.orchestra.run.smithi105.stdout:(17/28): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-01-24T19:24:23.358 INFO:teuthology.orchestra.run.smithi105.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2023-01-24T19:24:23.382 INFO:teuthology.orchestra.run.smithi105.stdout:(19/28): python3-trustme-0.6.0-4.el8.noarch.rpm 978 kB/s | 27 kB 00:00 2023-01-24T19:24:23.406 INFO:teuthology.orchestra.run.smithi105.stdout:(20/28): python3-cherrypy-18.4.0-1.el8.noarch.r 1.7 MB/s | 384 kB 00:00 2023-01-24T19:24:23.439 INFO:teuthology.orchestra.run.smithi105.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.6 MB/s | 85 kB 00:00 2023-01-24T19:24:23.474 INFO:teuthology.orchestra.run.smithi105.stdout:(22/28): python3-zc-lockfile-2.0-2.el8.noarch.r 662 kB/s | 23 kB 00:00 2023-01-24T19:24:23.477 INFO:teuthology.orchestra.run.smithi105.stdout:(23/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.6 MB/s | 251 kB 00:00 2023-01-24T19:24:23.478 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:24:23.480 INFO:teuthology.orchestra.run.smithi105.stdout:(24/28): python3-waitress-1.2.1-2.el8.1.noarch. 1.9 MB/s | 241 kB 00:00 2023-01-24T19:24:23.482 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:24:23.511 INFO:teuthology.orchestra.run.smithi105.stdout:(25/28): python3-werkzeug-0.12.2-4.el8.noarch.r 12 MB/s | 457 kB 00:00 2023-01-24T19:24:23.512 INFO:teuthology.orchestra.run.smithi105.stdout:(26/28): python3-markupsafe-0.23-19.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-01-24T19:24:23.520 INFO:teuthology.orchestra.run.smithi105.stdout:(27/28): python3-mako-1.0.6-13.el8.noarch.rpm 3.8 MB/s | 157 kB 00:00 2023-01-24T19:24:23.528 INFO:teuthology.orchestra.run.smithi105.stdout:(28/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.1 MB/s | 54 kB 00:00 2023-01-24T19:24:23.529 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:23.530 INFO:teuthology.orchestra.run.smithi105.stdout:Total 14 MB/s | 27 MB 00:01 2023-01-24T19:24:23.534 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:24:23.570 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:24:23.570 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:24:23.958 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:24:23.962 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:24:24.556 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:24:24.748 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T19:24:24.967 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T19:24:25.071 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:24:25.148 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T19:24:25.306 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T19:24:25.433 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T19:24:25.778 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T19:24:25.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T19:24:25.976 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T19:24:26.077 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T19:24:26.278 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T19:24:26.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T19:24:26.518 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T19:24:26.644 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T19:24:26.700 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T19:24:26.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T19:24:26.888 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T19:24:27.008 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T19:24:27.130 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T19:24:27.338 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T19:24:27.478 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T19:24:27.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T19:24:27.827 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T19:24:27.856 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T19:24:28.041 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T19:24:28.210 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T19:24:28.266 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T19:24:28.416 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T19:24:28.526 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T19:24:28.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T19:24:28.782 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T19:24:28.801 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T19:24:28.958 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T19:24:29.043 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T19:24:29.254 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T19:24:29.287 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T19:24:29.515 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T19:24:29.519 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T19:24:29.731 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T19:24:29.836 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T19:24:30.049 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T19:24:30.071 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T19:24:30.293 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T19:24:30.367 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T19:24:30.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T19:24:31.030 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T19:24:31.109 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T19:24:31.234 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T19:24:31.245 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T19:24:32.720 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T19:24:32.808 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T19:24:32.895 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T19:24:32.976 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T19:24:33.286 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T19:24:33.391 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T19:24:33.480 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T19:24:33.584 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T19:24:33.695 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T19:24:33.813 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T19:24:33.836 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T19:24:33.853 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T19:24:33.966 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T19:24:33.976 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T19:24:33.985 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T19:24:34.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T19:24:34.352 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T19:24:34.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T19:24:34.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T19:24:34.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T19:24:34.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T19:24:34.353 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T19:24:34.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T19:24:34.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T19:24:34.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T19:24:34.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T19:24:34.354 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T19:24:34.355 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T19:24:34.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T19:24:34.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T19:24:34.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T19:24:34.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T19:24:34.356 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T19:24:34.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T19:24:34.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T19:24:34.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T19:24:34.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T19:24:34.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T19:24:34.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T19:24:34.530 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T19:24:34.530 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T19:24:34.530 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T19:24:34.530 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T19:24:34.531 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T19:24:34.531 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T19:24:34.531 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T19:24:34.531 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T19:24:34.531 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T19:24:34.532 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T19:24:34.532 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T19:24:34.532 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T19:24:34.532 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T19:24:34.533 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T19:24:34.533 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T19:24:34.533 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T19:24:34.533 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T19:24:34.533 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T19:24:34.534 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T19:24:34.534 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T19:24:34.534 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T19:24:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T19:24:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T19:24:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T19:24:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T19:24:34.535 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T19:24:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T19:24:34.536 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T19:24:35.906 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T19:24:35.907 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:36.099 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T19:24:36.100 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:37.717 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:24:37.816 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:37.816 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:24:37.817 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:37.817 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:37.817 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:37.817 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:24:37.818 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:37.818 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:37.818 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T19:24:37.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T19:24:37.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T19:24:37.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T19:24:37.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T19:24:37.819 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T19:24:37.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T19:24:37.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T19:24:37.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T19:24:37.820 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T19:24:37.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T19:24:37.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T19:24:37.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T19:24:37.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T19:24:37.821 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T19:24:37.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T19:24:37.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T19:24:37.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T19:24:37.822 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T19:24:37.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T19:24:37.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T19:24:37.823 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T19:24:37.823 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:37.823 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:37.982 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:24:37.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-base 2023-01-24T19:24:38.094 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:38.095 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:24:38.096 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:38.096 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:38.096 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:38.097 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:24:38.097 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:38.097 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:38.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T19:24:38.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T19:24:38.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T19:24:38.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T19:24:38.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T19:24:38.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T19:24:38.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T19:24:38.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T19:24:38.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T19:24:38.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T19:24:38.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T19:24:38.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T19:24:38.102 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T19:24:38.102 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T19:24:38.102 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T19:24:38.103 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T19:24:38.103 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T19:24:38.103 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T19:24:38.104 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T19:24:38.104 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T19:24:38.104 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T19:24:38.105 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T19:24:38.105 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:38.105 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:24:38.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-base 2023-01-24T19:24:38.342 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:38.622 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:39.173 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:35 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:39.501 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:34 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:40.509 INFO:teuthology.orchestra.run.smithi105.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:24:40.867 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:24:40.959 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:24:40.960 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:24:40.961 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:41.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install cephadm 2023-01-24T19:24:41.344 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:24:41.345 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:24:41.345 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:24:41.427 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:41.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install cephadm 2023-01-24T19:24:41.845 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:42.267 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:38 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:42.706 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:37 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:44.067 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:24:44.068 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:44.068 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:24:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:24:44.069 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T19:24:44.069 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:44.069 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:24:44.070 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:44.070 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:24:44.070 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:44.070 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 78 k 2023-01-24T19:24:44.071 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 355 k 2023-01-24T19:24:44.071 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:24:44.212 INFO:teuthology.orchestra.run.smithi105.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 547 kB/s | 78 kB 00:00 2023-01-24T19:24:44.213 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:44.213 INFO:teuthology.orchestra.run.smithi105.stdout:Total 542 kB/s | 78 kB 00:00 2023-01-24T19:24:44.213 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:24:44.221 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:24:44.221 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:24:44.223 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:24:44.227 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:24:44.271 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:24:45.098 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:24:45.100 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:45.100 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:24:45.101 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:45.101 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:24:45.101 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T19:24:45.101 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:45.101 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:24:45.102 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:45.102 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:24:45.102 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:45.102 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 78 k 2023-01-24T19:24:45.102 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 355 k 2023-01-24T19:24:45.103 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:24:45.244 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 544 kB/s | 78 kB 00:00 2023-01-24T19:24:45.245 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:45.245 INFO:teuthology.orchestra.run.smithi062.stdout:Total 538 kB/s | 78 kB 00:00 2023-01-24T19:24:45.246 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:24:45.254 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:24:45.255 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:24:45.256 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:24:45.261 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:24:45.312 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:24:46.817 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:47.073 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:47.271 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:47.992 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:48.220 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:48.429 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:48.995 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:48.995 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:44 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:49.454 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:24:49.454 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:44 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:50.841 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:24:50.935 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:50.936 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:24:50.936 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:24:50.936 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:50.936 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:51.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T19:24:51.435 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:51.571 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:24:51.674 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:51.674 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:24:51.675 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:24:51.675 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:51.675 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:24:51.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T19:24:52.198 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:52.272 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:53.153 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:48 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:54.048 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:24:54.049 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:54.049 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T19:24:54.049 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:54.049 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:24:54.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:54.051 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 162 k 2023-01-24T19:24:54.051 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 490 k 2023-01-24T19:24:54.051 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:24:54.220 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 952 kB/s | 162 kB 00:00 2023-01-24T19:24:54.221 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:54.221 INFO:teuthology.orchestra.run.smithi105.stdout:Total 945 kB/s | 162 kB 00:00 2023-01-24T19:24:54.221 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:24:54.233 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:24:54.233 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:24:54.284 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:24:54.289 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:24:54.406 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:24:54.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:55.005 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:55.083 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:24:55.085 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:55.085 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-24T19:24:55.085 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:55.086 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:24:55.086 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T19:24:55.086 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:55.087 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:24:55.087 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:24:55.087 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:24:55.087 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:55.087 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 162 k 2023-01-24T19:24:55.088 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 490 k 2023-01-24T19:24:55.088 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:24:55.295 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 769 kB/s | 162 kB 00:00 2023-01-24T19:24:55.296 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:24:55.296 INFO:teuthology.orchestra.run.smithi062.stdout:Total 764 kB/s | 162 kB 00:00 2023-01-24T19:24:55.296 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:24:55.314 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:24:55.314 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:24:55.367 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:24:55.371 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:24:55.495 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:24:55.696 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:56.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:56.071 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:56.159 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:57.221 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:24:57.221 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:52 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:24:57.911 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:24:58.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:58.007 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:24:58.008 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:58.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:24:58.008 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:24:58.128 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr 2023-01-24T19:24:58.485 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:24:59.158 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:24:59.266 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:59.266 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:24:59.267 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:24:59.267 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:24:59.267 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:24:59.334 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:24:59.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr 2023-01-24T19:24:59.763 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:00.638 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:00.760 INFO:teuthology.orchestra.run.smithi105.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:25:01.211 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:01.212 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:25:01.212 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:01.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T19:25:01.668 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:02.089 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:25:02.506 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:58 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:02.572 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:02.573 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:25:02.573 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:02.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T19:25:03.053 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:03.942 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:58 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:04.383 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:04.384 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================================== 2023-01-24T19:25:04.384 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:25:04.384 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================================== 2023-01-24T19:25:04.384 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:25:04.385 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T19:25:04.385 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:25:04.385 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T19:25:04.385 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T19:25:04.385 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T19:25:04.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T19:25:04.386 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T19:25:04.386 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:04.386 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:25:04.386 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================================== 2023-01-24T19:25:04.387 INFO:teuthology.orchestra.run.smithi105.stdout:Install 6 Packages 2023-01-24T19:25:04.387 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:04.388 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 2.0 M 2023-01-24T19:25:04.388 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 7.8 M 2023-01-24T19:25:04.388 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:25:04.637 INFO:teuthology.orchestra.run.smithi105.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 92 kB/s | 23 kB 00:00 2023-01-24T19:25:04.637 INFO:teuthology.orchestra.run.smithi105.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 58 kB/s | 14 kB 00:00 2023-01-24T19:25:04.755 INFO:teuthology.orchestra.run.smithi105.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 286 kB/s | 34 kB 00:00 2023-01-24T19:25:04.815 INFO:teuthology.orchestra.run.smithi105.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 707 kB/s | 43 kB 00:00 2023-01-24T19:25:04.848 INFO:teuthology.orchestra.run.smithi105.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 933 kB/s | 196 kB 00:00 2023-01-24T19:25:05.020 INFO:teuthology.orchestra.run.smithi105.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 2.6 MB/s | 1.6 MB 00:00 2023-01-24T19:25:05.023 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:05.023 INFO:teuthology.orchestra.run.smithi105.stdout:Total 3.1 MB/s | 2.0 MB 00:00 2023-01-24T19:25:05.025 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:25:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:25:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:25:05.119 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:25:05.124 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:25:05.387 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:25:05.618 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T19:25:05.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T19:25:05.907 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:05.908 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================================== 2023-01-24T19:25:05.908 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:25:05.908 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================================== 2023-01-24T19:25:05.908 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:25:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T19:25:05.909 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:25:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T19:25:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T19:25:05.909 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T19:25:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T19:25:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T19:25:05.910 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:05.910 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:25:05.910 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================================== 2023-01-24T19:25:05.911 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2023-01-24T19:25:05.911 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:05.911 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 2.0 M 2023-01-24T19:25:05.912 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 7.8 M 2023-01-24T19:25:05.912 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:25:06.018 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T19:25:06.117 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 111 kB/s | 23 kB 00:00 2023-01-24T19:25:06.129 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T19:25:06.172 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 55 kB/s | 14 kB 00:00 2023-01-24T19:25:06.310 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 174 kB/s | 34 kB 00:00 2023-01-24T19:25:06.389 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 543 kB/s | 43 kB 00:00 2023-01-24T19:25:06.438 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 741 kB/s | 196 kB 00:00 2023-01-24T19:25:06.500 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T19:25:06.601 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 2.4 MB/s | 1.6 MB 00:00 2023-01-24T19:25:06.604 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:06.604 INFO:teuthology.orchestra.run.smithi062.stdout:Total 2.8 MB/s | 2.0 MB 00:00 2023-01-24T19:25:06.606 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:25:06.616 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:25:06.617 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:25:06.714 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:25:06.719 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:25:06.748 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:25:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:25:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T19:25:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T19:25:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T19:25:07.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T19:25:07.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T19:25:07.100 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:25:07.320 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T19:25:07.500 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T19:25:07.690 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T19:25:07.819 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T19:25:08.084 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T19:25:08.085 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:04 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:08.231 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T19:25:08.454 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:25:08.740 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:25:08.740 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T19:25:08.740 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T19:25:08.741 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T19:25:08.741 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T19:25:08.741 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T19:25:09.816 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T19:25:09.817 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:04 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:10.128 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:25:10.226 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:10.227 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:25:10.227 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:10.227 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:10.227 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:10.227 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T19:25:10.228 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T19:25:10.228 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T19:25:10.228 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:10.228 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:10.340 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T19:25:10.698 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:11.561 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:07 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:11.852 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:25:11.960 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:11.960 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:25:11.960 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:11.961 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:11.961 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:11.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T19:25:11.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T19:25:11.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T19:25:11.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:11.962 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:12.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T19:25:12.466 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:13.407 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:13.408 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================================== 2023-01-24T19:25:13.408 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:25:13.408 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================================== 2023-01-24T19:25:13.409 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:25:13.409 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T19:25:13.409 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:25:13.409 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T19:25:13.409 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T19:25:13.410 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T19:25:13.410 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T19:25:13.410 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T19:25:13.410 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T19:25:13.410 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T19:25:13.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:13.411 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:25:13.411 INFO:teuthology.orchestra.run.smithi105.stdout:====================================================================================================================== 2023-01-24T19:25:13.411 INFO:teuthology.orchestra.run.smithi105.stdout:Install 8 Packages 2023-01-24T19:25:13.411 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:13.412 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:08 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:13.413 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 36 M 2023-01-24T19:25:13.413 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 199 M 2023-01-24T19:25:13.413 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:25:13.470 INFO:teuthology.orchestra.run.smithi105.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 3.7 MB/s | 225 kB 00:00 2023-01-24T19:25:13.539 INFO:teuthology.orchestra.run.smithi105.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 54 MB/s | 3.7 MB 00:00 2023-01-24T19:25:13.709 INFO:teuthology.orchestra.run.smithi105.stdout:(3/8): python3-scipy-1.0.0-20.module+el8.1.0+33 49 MB/s | 15 MB 00:00 2023-01-24T19:25:13.726 INFO:teuthology.orchestra.run.smithi105.stdout:(4/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 25 MB/s | 4.7 MB 00:00 2023-01-24T19:25:13.742 INFO:teuthology.orchestra.run.smithi105.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-01-24T19:25:13.762 INFO:teuthology.orchestra.run.smithi105.stdout:(6/8): libgfortran-8.4.1-1.el8.x86_64.rpm 32 MB/s | 639 kB 00:00 2023-01-24T19:25:13.785 INFO:teuthology.orchestra.run.smithi105.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 61 MB/s | 4.6 MB 00:00 2023-01-24T19:25:15.285 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:15.286 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-01-24T19:25:15.286 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:25:15.286 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-01-24T19:25:15.287 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:25:15.287 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T19:25:15.287 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:25:15.287 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T19:25:15.287 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T19:25:15.288 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T19:25:15.288 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T19:25:15.288 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T19:25:15.288 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout:====================================================================================================================== 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout:Install 8 Packages 2023-01-24T19:25:15.289 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:15.290 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 36 M 2023-01-24T19:25:15.290 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 199 M 2023-01-24T19:25:15.291 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:25:15.332 INFO:teuthology.orchestra.run.smithi062.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 5.1 MB/s | 225 kB 00:00 2023-01-24T19:25:15.406 INFO:teuthology.orchestra.run.smithi062.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 50 MB/s | 3.7 MB 00:00 2023-01-24T19:25:15.490 INFO:teuthology.orchestra.run.smithi062.stdout:(3/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 56 MB/s | 4.7 MB 00:00 2023-01-24T19:25:15.589 INFO:teuthology.orchestra.run.smithi105.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 3.4 MB/s | 7.4 MB 00:02 2023-01-24T19:25:15.591 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:15.592 INFO:teuthology.orchestra.run.smithi105.stdout:Total 16 MB/s | 36 MB 00:02 2023-01-24T19:25:15.602 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:25:15.621 INFO:teuthology.orchestra.run.smithi062.stdout:(4/8): python3-scipy-1.0.0-20.module+el8.1.0+33 44 MB/s | 15 MB 00:00 2023-01-24T19:25:15.622 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:25:15.622 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:25:15.636 INFO:teuthology.orchestra.run.smithi062.stdout:(5/8): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-01-24T19:25:15.659 INFO:teuthology.orchestra.run.smithi062.stdout:(6/8): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-01-24T19:25:15.678 INFO:teuthology.orchestra.run.smithi062.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 24 MB/s | 4.6 MB 00:00 2023-01-24T19:25:16.054 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:25:16.058 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:25:16.702 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:25:16.855 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T19:25:16.972 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T19:25:17.121 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T19:25:17.333 INFO:teuthology.orchestra.run.smithi062.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 3.6 MB/s | 7.4 MB 00:02 2023-01-24T19:25:17.335 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:17.336 INFO:teuthology.orchestra.run.smithi062.stdout:Total 18 MB/s | 36 MB 00:02 2023-01-24T19:25:17.346 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:25:17.368 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:25:17.369 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:25:17.658 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T19:25:17.791 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:25:17.795 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:25:17.854 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T19:25:18.372 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T19:25:18.474 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:25:18.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T19:25:18.652 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T19:25:18.796 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T19:25:18.935 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T19:25:19.298 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T19:25:19.516 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T19:25:19.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T19:25:19.660 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T19:25:19.854 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T19:25:20.225 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T19:25:20.376 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T19:25:21.167 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T19:25:21.465 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T19:25:21.668 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T19:25:21.916 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T19:25:23.183 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T19:25:23.362 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:25:23.660 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:25:23.660 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T19:25:23.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-01-24T19:25:23.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-01-24T19:25:23.661 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-01-24T19:25:23.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-01-24T19:25:23.662 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-01-24T19:25:23.663 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-01-24T19:25:23.864 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T19:25:24.693 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-01-24T19:25:24.693 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:20 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:25.158 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T19:25:25.342 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:25:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:25:25.656 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T19:25:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 2/8 2023-01-24T19:25:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2023-01-24T19:25:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2023-01-24T19:25:25.657 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 5/8 2023-01-24T19:25:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 6/8 2023-01-24T19:25:25.658 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 7/8 2023-01-24T19:25:26.651 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:25:26.749 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:26.749 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:25:26.750 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:26.750 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T19:25:26.751 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T19:25:26.751 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T19:25:26.751 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T19:25:26.752 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T19:25:26.752 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T19:25:26.752 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T19:25:26.753 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:26.753 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:26.754 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 8/8 2023-01-24T19:25:26.754 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:21 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:26.896 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-rook 2023-01-24T19:25:27.255 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:28.095 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:24 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:28.697 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:25:28.808 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:28.808 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:25:28.808 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:28.809 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T19:25:28.809 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T19:25:28.809 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T19:25:28.809 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T19:25:28.809 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T19:25:28.810 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T19:25:28.810 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T19:25:28.810 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:28.810 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:28.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-rook 2023-01-24T19:25:29.330 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:29.902 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:29.904 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================================= 2023-01-24T19:25:29.904 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:25:29.904 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================================= 2023-01-24T19:25:29.905 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:25:29.905 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T19:25:29.905 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:25:29.905 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T19:25:29.905 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T19:25:29.906 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T19:25:29.906 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T19:25:29.906 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T19:25:29.906 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T19:25:29.906 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T19:25:29.907 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T19:25:29.907 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T19:25:29.907 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T19:25:29.907 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T19:25:29.907 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T19:25:29.908 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:29.908 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:25:29.908 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================================= 2023-01-24T19:25:29.908 INFO:teuthology.orchestra.run.smithi105.stdout:Install 13 Packages 2023-01-24T19:25:29.908 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:29.909 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 1.9 M 2023-01-24T19:25:29.909 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 25 M 2023-01-24T19:25:29.910 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:25:30.021 INFO:teuthology.orchestra.run.smithi105.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 143 kB/s | 16 kB 00:00 2023-01-24T19:25:30.022 INFO:teuthology.orchestra.run.smithi105.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 286 kB/s | 33 kB 00:00 2023-01-24T19:25:30.102 INFO:teuthology.orchestra.run.smithi105.stdout:(3/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 240 kB/s | 47 kB 00:00 2023-01-24T19:25:30.120 INFO:teuthology.orchestra.run.smithi105.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 823 kB/s | 81 kB 00:00 2023-01-24T19:25:30.171 INFO:teuthology.orchestra.run.smithi105.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-24T19:25:30.213 INFO:teuthology.orchestra.run.smithi105.stdout:(6/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 640 kB/s | 27 kB 00:00 2023-01-24T19:25:30.219 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:25 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:30.238 INFO:teuthology.orchestra.run.smithi105.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.9 MB/s | 126 kB 00:00 2023-01-24T19:25:30.264 INFO:teuthology.orchestra.run.smithi105.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 4.2 MB/s | 110 kB 00:00 2023-01-24T19:25:30.277 INFO:teuthology.orchestra.run.smithi105.stdout:(9/13): python3-rsa-4.8-1.el8.noarch.rpm 353 kB/s | 61 kB 00:00 2023-01-24T19:25:30.290 INFO:teuthology.orchestra.run.smithi105.stdout:(10/13): python3-jsonpointer-1.10-11.el8.noarch 771 kB/s | 20 kB 00:00 2023-01-24T19:25:30.303 INFO:teuthology.orchestra.run.smithi105.stdout:(11/13): python3-requests-oauthlib-1.0.0-1.el8. 3.2 MB/s | 43 kB 00:00 2023-01-24T19:25:30.304 INFO:teuthology.orchestra.run.smithi105.stdout:(12/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 5.6 MB/s | 155 kB 00:00 2023-01-24T19:25:30.311 INFO:teuthology.orchestra.run.smithi105.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 4.0 MB/s | 1.2 MB 00:00 2023-01-24T19:25:30.312 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:30.312 INFO:teuthology.orchestra.run.smithi105.stdout:Total 4.7 MB/s | 1.9 MB 00:00 2023-01-24T19:25:30.317 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:25:30.333 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:25:30.334 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:25:30.414 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:25:30.419 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:25:30.915 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:25:31.215 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T19:25:31.500 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T19:25:31.787 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T19:25:32.093 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:32.095 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================================= 2023-01-24T19:25:32.095 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:25:32.096 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================================= 2023-01-24T19:25:32.096 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:25:32.097 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T19:25:32.097 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:25:32.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T19:25:32.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T19:25:32.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T19:25:32.098 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T19:25:32.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T19:25:32.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T19:25:32.099 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T19:25:32.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T19:25:32.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T19:25:32.100 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T19:25:32.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T19:25:32.101 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T19:25:32.101 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:32.102 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:25:32.102 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================================= 2023-01-24T19:25:32.102 INFO:teuthology.orchestra.run.smithi062.stdout:Install 13 Packages 2023-01-24T19:25:32.102 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:32.103 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-01-24T19:25:32.103 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 1.9 M 2023-01-24T19:25:32.104 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 25 M 2023-01-24T19:25:32.104 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:25:32.270 INFO:teuthology.orchestra.run.smithi062.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 95 kB/s | 16 kB 00:00 2023-01-24T19:25:32.273 INFO:teuthology.orchestra.run.smithi062.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 188 kB/s | 33 kB 00:00 2023-01-24T19:25:32.343 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-01-24T19:25:32.409 INFO:teuthology.orchestra.run.smithi062.stdout:(3/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 151 kB/s | 47 kB 00:00 2023-01-24T19:25:32.429 INFO:teuthology.orchestra.run.smithi062.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 511 kB/s | 81 kB 00:00 2023-01-24T19:25:32.477 INFO:teuthology.orchestra.run.smithi062.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-24T19:25:32.512 INFO:teuthology.orchestra.run.smithi062.stdout:(6/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 789 kB/s | 27 kB 00:00 2023-01-24T19:25:32.528 INFO:teuthology.orchestra.run.smithi062.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.8 MB/s | 126 kB 00:00 2023-01-24T19:25:32.540 INFO:teuthology.orchestra.run.smithi062.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 7.4 MB/s | 110 kB 00:00 2023-01-24T19:25:32.553 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-01-24T19:25:32.553 INFO:teuthology.orchestra.run.smithi062.stdout:(9/13): python3-jsonpointer-1.10-11.el8.noarch. 1.6 MB/s | 20 kB 00:00 2023-01-24T19:25:32.567 INFO:teuthology.orchestra.run.smithi062.stdout:(10/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 11 MB/s | 155 kB 00:00 2023-01-24T19:25:32.579 INFO:teuthology.orchestra.run.smithi062.stdout:(11/13): python3-requests-oauthlib-1.0.0-1.el8. 3.5 MB/s | 43 kB 00:00 2023-01-24T19:25:32.621 INFO:teuthology.orchestra.run.smithi062.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 291 kB/s | 61 kB 00:00 2023-01-24T19:25:32.649 INFO:teuthology.orchestra.run.smithi062.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.1 MB/s | 1.2 MB 00:00 2023-01-24T19:25:32.650 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:32.650 INFO:teuthology.orchestra.run.smithi062.stdout:Total 3.5 MB/s | 1.9 MB 00:00 2023-01-24T19:25:32.656 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:25:32.671 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:25:32.671 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:25:32.746 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-01-24T19:25:32.767 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:25:32.771 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:25:33.046 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T19:25:33.290 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T19:25:33.412 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:25:33.594 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T19:25:33.682 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T19:25:33.918 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T19:25:34.193 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T19:25:34.460 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-01-24T19:25:34.708 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-01-24T19:25:34.795 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T19:25:34.910 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-01-24T19:25:35.105 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T19:25:35.129 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-01-24T19:25:35.285 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:25:35.386 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T19:25:35.539 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:25:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T19:25:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T19:25:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T19:25:35.540 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T19:25:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T19:25:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T19:25:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T19:25:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-01-24T19:25:35.541 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-01-24T19:25:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-24T19:25:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T19:25:35.542 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-01-24T19:25:35.572 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T19:25:35.846 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T19:25:36.612 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-01-24T19:25:36.612 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:32 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:37.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T19:25:37.428 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T19:25:37.574 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:25:37.877 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:25:37.877 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T19:25:37.878 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T19:25:37.878 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T19:25:37.878 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T19:25:37.878 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 9/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-24T19:25:37.879 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T19:25:37.880 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-01-24T19:25:38.448 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:25:38.545 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:38.546 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:25:38.546 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:38.546 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T19:25:38.546 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T19:25:38.547 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T19:25:38.547 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T19:25:38.547 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T19:25:38.547 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T19:25:38.547 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T19:25:38.548 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T19:25:38.548 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T19:25:38.548 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T19:25:38.548 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T19:25:38.548 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T19:25:38.549 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:38.549 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:38.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T19:25:39.045 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:39.075 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-01-24T19:25:39.075 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:34 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:39.882 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:41.007 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:25:41.114 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:41.114 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:25:41.114 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:41.115 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T19:25:41.115 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T19:25:41.115 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T19:25:41.115 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T19:25:41.115 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T19:25:41.116 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T19:25:41.116 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T19:25:41.116 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T19:25:41.116 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T19:25:41.117 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T19:25:41.117 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T19:25:41.117 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T19:25:41.117 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:41.117 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:41.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T19:25:41.639 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:41.714 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:41.715 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================== 2023-01-24T19:25:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:25:41.716 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================== 2023-01-24T19:25:41.716 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:25:41.716 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T19:25:41.716 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:25:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T19:25:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T19:25:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T19:25:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T19:25:41.717 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:41.718 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:25:41.718 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================== 2023-01-24T19:25:41.718 INFO:teuthology.orchestra.run.smithi105.stdout:Install 5 Packages 2023-01-24T19:25:41.718 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:41.719 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 5.9 M 2023-01-24T19:25:41.719 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 26 M 2023-01-24T19:25:41.719 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:25:41.885 INFO:teuthology.orchestra.run.smithi105.stdout:(1/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 687 kB/s | 115 kB 00:00 2023-01-24T19:25:41.913 INFO:teuthology.orchestra.run.smithi105.stdout:(2/5): python3-natsort-7.1.1-2.el8.noarch.rpm 309 kB/s | 60 kB 00:00 2023-01-24T19:25:41.961 INFO:teuthology.orchestra.run.smithi105.stdout:(3/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 11 MB/s | 538 kB 00:00 2023-01-24T19:25:41.981 INFO:teuthology.orchestra.run.smithi105.stdout:(4/5): python3-babel-2.5.1-5.el8.noarch.rpm 50 MB/s | 4.8 MB 00:00 2023-01-24T19:25:42.042 INFO:teuthology.orchestra.run.smithi105.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2023-01-24T19:25:42.043 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:42.044 INFO:teuthology.orchestra.run.smithi105.stdout:Total 18 MB/s | 5.9 MB 00:00 2023-01-24T19:25:42.047 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:25:42.057 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:25:42.057 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:25:42.151 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:25:42.155 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:25:42.546 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:37 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:43.172 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:25:43.564 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T19:25:43.830 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T19:25:44.183 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T19:25:44.431 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:44.431 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================== 2023-01-24T19:25:44.432 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:25:44.432 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================== 2023-01-24T19:25:44.432 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:25:44.432 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T19:25:44.433 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:25:44.433 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T19:25:44.433 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T19:25:44.433 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T19:25:44.433 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T19:25:44.434 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:25:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:============================================================================================== 2023-01-24T19:25:44.434 INFO:teuthology.orchestra.run.smithi062.stdout:Install 5 Packages 2023-01-24T19:25:44.434 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 5.9 M 2023-01-24T19:25:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 26 M 2023-01-24T19:25:44.435 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:25:44.447 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T19:25:44.543 INFO:teuthology.orchestra.run.smithi062.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 553 kB/s | 60 kB 00:00 2023-01-24T19:25:44.601 INFO:teuthology.orchestra.run.smithi062.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 690 kB/s | 115 kB 00:00 2023-01-24T19:25:44.623 INFO:teuthology.orchestra.run.smithi062.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.5 MB/s | 476 kB 00:00 2023-01-24T19:25:44.643 INFO:teuthology.orchestra.run.smithi062.stdout:(4/5): python3-babel-2.5.1-5.el8.noarch.rpm 48 MB/s | 4.8 MB 00:00 2023-01-24T19:25:44.647 INFO:teuthology.orchestra.run.smithi062.stdout:(5/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 11 MB/s | 538 kB 00:00 2023-01-24T19:25:44.649 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:44.649 INFO:teuthology.orchestra.run.smithi062.stdout:Total 27 MB/s | 5.9 MB 00:00 2023-01-24T19:25:44.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:25:44.653 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:25:44.664 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:25:44.664 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:25:44.761 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:25:44.765 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:25:44.895 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:25:44.895 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T19:25:44.896 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T19:25:44.896 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T19:25:44.896 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T19:25:45.810 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:25:45.903 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T19:25:45.903 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:41 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:46.205 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T19:25:46.478 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T19:25:46.850 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T19:25:47.113 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T19:25:47.298 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:25:47.578 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:25:47.578 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T19:25:47.578 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T19:25:47.579 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T19:25:47.579 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T19:25:47.816 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:25:47.914 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:47.914 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:25:47.914 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:47.914 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T19:25:47.915 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T19:25:47.915 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T19:25:47.915 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T19:25:47.915 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:47.915 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:48.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-fuse 2023-01-24T19:25:48.385 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:48.643 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T19:25:48.643 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:43 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:49.282 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:45 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:50.562 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:25:50.672 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:50.672 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:25:50.672 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:25:50.672 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T19:25:50.673 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T19:25:50.673 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T19:25:50.673 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T19:25:50.673 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:50.673 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:50.793 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-fuse 2023-01-24T19:25:51.152 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:25:51.152 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:25:51.152 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:25:51.153 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:25:51.153 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:25:51.153 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse 2023-01-24T19:25:51.153 INFO:teuthology.orchestra.run.smithi105.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T19:25:51.153 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T19:25:51.154 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T19:25:51.154 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:51.154 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:25:51.154 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:25:51.154 INFO:teuthology.orchestra.run.smithi105.stdout:Install 2 Packages 2023-01-24T19:25:51.155 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:51.155 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 954 k 2023-01-24T19:25:51.155 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 3.0 M 2023-01-24T19:25:51.156 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:25:51.182 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:51.183 INFO:teuthology.orchestra.run.smithi105.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.8 MB/s | 83 kB 00:00 2023-01-24T19:25:51.481 INFO:teuthology.orchestra.run.smithi105.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 2.6 MB/s | 872 kB 00:00 2023-01-24T19:25:51.482 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:51.482 INFO:teuthology.orchestra.run.smithi105.stdout:Total 2.8 MB/s | 954 kB 00:00 2023-01-24T19:25:51.484 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:25:51.505 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:25:51.505 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:25:51.587 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:25:51.591 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:25:51.758 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:25:52.050 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:47 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T19:25:54.361 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:25:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:25:54.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T19:25:54.801 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:25:54.802 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:25:54.802 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:25:54.802 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:25:54.802 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:25:54.802 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse 2023-01-24T19:25:54.803 INFO:teuthology.orchestra.run.smithi062.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T19:25:54.803 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-24T19:25:54.803 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T19:25:54.803 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:54.803 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:25:54.804 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:25:54.804 INFO:teuthology.orchestra.run.smithi062.stdout:Install 2 Packages 2023-01-24T19:25:54.804 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:54.804 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 954 k 2023-01-24T19:25:54.804 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 3.0 M 2023-01-24T19:25:54.805 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:25:54.835 INFO:teuthology.orchestra.run.smithi062.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 2.5 MB/s | 83 kB 00:00 2023-01-24T19:25:55.156 INFO:teuthology.orchestra.run.smithi062.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 2.4 MB/s | 872 kB 00:00 2023-01-24T19:25:55.157 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:25:55.158 INFO:teuthology.orchestra.run.smithi062.stdout:Total 2.6 MB/s | 954 kB 00:00 2023-01-24T19:25:55.159 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:25:55.182 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:25:55.183 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:25:55.271 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:25:55.275 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:25:55.449 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:25:55.736 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T19:25:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T19:25:55.762 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:51 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:55.939 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:25:56.340 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:25:56.340 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T19:25:57.409 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T19:25:57.409 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:52 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:25:57.874 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:25:57.971 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:57.972 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:25:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T19:25:57.972 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:25:57.972 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:25:58.082 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-volume 2023-01-24T19:25:58.450 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:25:59.313 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:55 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:25:59.518 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:25:59.623 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:59.623 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:25:59.623 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T19:25:59.624 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:25:59.624 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:25:59.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-volume 2023-01-24T19:26:00.134 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:01.015 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:56 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:01.074 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:01.075 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:01.075 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T19:26:01.075 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:01.075 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:01.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T19:26:01.076 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:01.076 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:01.076 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:01.076 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:01.076 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:01.077 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 263 k 2023-01-24T19:26:01.077 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 1.3 M 2023-01-24T19:26:01.077 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:01.327 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.0 MB/s | 263 kB 00:00 2023-01-24T19:26:01.328 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:01.329 INFO:teuthology.orchestra.run.smithi105.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-24T19:26:01.329 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:01.339 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:01.339 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:01.360 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:01.365 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:01.541 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:01.841 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:02.261 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:02.911 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:02.912 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-24T19:26:02.912 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:26:02.913 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T19:26:02.913 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:26:02.913 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:26:02.914 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 263 k 2023-01-24T19:26:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 1.3 M 2023-01-24T19:26:02.914 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:26:03.119 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.2 MB/s | 263 kB 00:00 2023-01-24T19:26:03.119 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:03.120 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-01-24T19:26:03.120 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:26:03.131 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:26:03.132 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:26:03.161 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:26:03.165 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:26:03.269 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:03.269 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:02:59 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:03.385 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:26:03.663 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:04.098 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:05.144 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:26:05.145 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:00 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:05.205 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:26:05.300 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:05.300 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:26:05.301 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:26:05.301 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:05.301 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:05.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librados-devel 2023-01-24T19:26:05.783 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:06.638 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:02 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:07.182 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:26:07.286 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:26:07.287 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:26:07.287 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:07.287 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:07.400 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados-devel 2023-01-24T19:26:07.794 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:08.426 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:08.427 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:08.427 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T19:26:08.427 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:08.427 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T19:26:08.428 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:08.428 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:08.428 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:08.428 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:08.429 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:08.429 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 130 k 2023-01-24T19:26:08.429 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 464 k 2023-01-24T19:26:08.429 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:08.597 INFO:teuthology.orchestra.run.smithi105.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 764 kB/s | 130 kB 00:00 2023-01-24T19:26:08.598 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:08.598 INFO:teuthology.orchestra.run.smithi105.stdout:Total 757 kB/s | 130 kB 00:00 2023-01-24T19:26:08.599 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:08.605 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:08.605 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:08.661 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:08.665 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:08.666 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:03 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:08.789 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:09.005 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:09.269 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:10.316 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:10.317 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:06 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:10.546 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:10.547 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:10.547 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-24T19:26:10.547 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:10.547 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:26:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T19:26:10.548 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:10.548 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:26:10.548 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:10.549 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:26:10.549 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:10.549 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 130 k 2023-01-24T19:26:10.549 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 464 k 2023-01-24T19:26:10.549 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:26:10.719 INFO:teuthology.orchestra.run.smithi062.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 760 kB/s | 130 kB 00:00 2023-01-24T19:26:10.719 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:10.720 INFO:teuthology.orchestra.run.smithi062.stdout:Total 753 kB/s | 130 kB 00:00 2023-01-24T19:26:10.720 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:26:10.727 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:26:10.727 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:26:10.787 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:26:10.792 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:26:10.963 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:26:11.169 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:11.448 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:12.298 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:26:12.396 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:12.396 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:26:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:12.397 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:12.397 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:12.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install libcephfs2 2023-01-24T19:26:12.540 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:12.540 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:07 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:12.896 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:13.775 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:09 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:14.459 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:26:14.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:14.566 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:26:14.566 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:14.566 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:14.567 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:14.691 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs2 2023-01-24T19:26:15.083 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:15.200 INFO:teuthology.orchestra.run.smithi105.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:15.651 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:15.652 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:15.652 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:15.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install libcephfs-devel 2023-01-24T19:26:15.962 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:10 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:16.112 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:16.969 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:12 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:17.471 INFO:teuthology.orchestra.run.smithi062.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:17.949 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:17.950 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:18.044 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs-devel 2023-01-24T19:26:18.439 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:18.831 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:18.832 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:18.832 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T19:26:18.833 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:18.833 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:18.833 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T19:26:18.833 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:18.833 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:18.834 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:18.834 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:18.834 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:18.834 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 25 k 2023-01-24T19:26:18.834 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 109 k 2023-01-24T19:26:18.835 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:18.956 INFO:teuthology.orchestra.run.smithi105.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 202 kB/s | 25 kB 00:00 2023-01-24T19:26:18.956 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:18.957 INFO:teuthology.orchestra.run.smithi105.stdout:Total 199 kB/s | 25 kB 00:00 2023-01-24T19:26:18.957 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:18.960 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:18.961 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:18.962 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:18.967 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:19.020 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:19.187 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:19.354 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:14 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:19.424 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:20.429 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:20.429 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:16 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:21.271 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:21.272 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-24T19:26:21.273 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:21.273 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:26:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T19:26:21.273 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:21.274 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:26:21.274 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:21.274 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:26:21.274 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:21.274 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 25 k 2023-01-24T19:26:21.275 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 109 k 2023-01-24T19:26:21.275 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:26:21.410 INFO:teuthology.orchestra.run.smithi062.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 182 kB/s | 25 kB 00:00 2023-01-24T19:26:21.410 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:21.411 INFO:teuthology.orchestra.run.smithi062.stdout:Total 180 kB/s | 25 kB 00:00 2023-01-24T19:26:21.411 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:26:21.415 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:26:21.415 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:26:21.417 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:26:21.421 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:26:21.471 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:26:21.634 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:21.910 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:22.307 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:26:22.402 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:22.402 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:26:22.403 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:22.403 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:22.403 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:22.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librados2 2023-01-24T19:26:22.878 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:22.939 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:22.939 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:17 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:23.711 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:19 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:25.062 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:25.146 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:26:25.253 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:25.254 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:26:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:25.254 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:25.254 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:25.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados2 2023-01-24T19:26:25.515 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:25.516 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:25.516 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:25.608 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librbd1 2023-01-24T19:26:25.783 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:25.973 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:26.678 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:21 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:26.820 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:22 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:28.046 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:28.195 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:28.515 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:28.516 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:28.516 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:28.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librbd1 2023-01-24T19:26:28.641 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:28.642 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:28.642 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:28.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rados 2023-01-24T19:26:29.001 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:29.112 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:29.881 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:24 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:30.016 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:26 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:31.319 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:31.403 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:31.801 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:31.802 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:31.803 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:31.866 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:31.867 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:31.867 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:31.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rados 2023-01-24T19:26:31.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rgw 2023-01-24T19:26:32.296 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:32.344 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:33.166 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:28 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:33.177 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:29 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:34.494 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:34.577 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:34.947 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:34.948 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:34.949 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:35.051 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:35.052 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:35.052 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:35.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-cephfs 2023-01-24T19:26:35.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rgw 2023-01-24T19:26:35.423 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:35.541 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:36.309 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:32 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:36.408 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:31 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:37.652 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:37.793 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:38.098 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:38.098 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:38.099 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:38.195 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rbd 2023-01-24T19:26:38.275 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:38.276 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:38.276 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:38.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-cephfs 2023-01-24T19:26:38.563 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:38.769 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:39.406 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:35 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:39.658 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:34 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:40.727 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:41.092 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:41.185 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:41.186 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T19:26:41.187 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:41.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-fuse 2023-01-24T19:26:41.563 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:41.564 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:41.564 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:41.650 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:41.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rbd 2023-01-24T19:26:42.062 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:42.485 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:38 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:42.934 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:37 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:44.257 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:44.258 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:44.258 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2023-01-24T19:26:44.259 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:44.259 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:44.259 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T19:26:44.259 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:44.259 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:44.260 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:44.260 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:44.260 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:44.260 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 90 k 2023-01-24T19:26:44.260 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 264 k 2023-01-24T19:26:44.261 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:44.384 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:26:44.402 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 630 kB/s | 90 kB 00:00 2023-01-24T19:26:44.402 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:44.403 INFO:teuthology.orchestra.run.smithi105.stdout:Total 624 kB/s | 90 kB 00:00 2023-01-24T19:26:44.403 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:44.415 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:44.415 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:44.470 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:44.474 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:44.593 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:44.826 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:44.862 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:44.863 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-24T19:26:44.864 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:44.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-fuse 2023-01-24T19:26:45.100 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:45.352 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:46.116 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:46.117 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:42 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:46.232 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:41 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:47.979 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:26:48.074 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:48.075 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:26:48.075 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:48.075 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:48.075 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:48.159 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:48.160 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:48.160 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-01-24T19:26:48.160 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:48.161 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:26:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T19:26:48.161 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:48.161 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:26:48.161 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:48.162 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:26:48.162 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:48.162 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 90 k 2023-01-24T19:26:48.162 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 264 k 2023-01-24T19:26:48.162 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:26:48.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-mirror 2023-01-24T19:26:48.304 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 630 kB/s | 90 kB 00:00 2023-01-24T19:26:48.305 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:48.305 INFO:teuthology.orchestra.run.smithi062.stdout:Total 624 kB/s | 90 kB 00:00 2023-01-24T19:26:48.305 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:26:48.318 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:26:48.318 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:26:48.377 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:26:48.381 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:26:48.504 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:26:48.551 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:48.714 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:48.988 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:49.390 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:45 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:50.058 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:50.058 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:45 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:51.206 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:51.207 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:51.207 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T19:26:51.207 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:51.208 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:51.208 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:26:51.208 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:51.208 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 3.5 M 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 16 M 2023-01-24T19:26:51.209 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:51.608 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 8.6 MB/s | 3.5 MB 00:00 2023-01-24T19:26:51.608 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:51.609 INFO:teuthology.orchestra.run.smithi105.stdout:Total 8.6 MB/s | 3.5 MB 00:00 2023-01-24T19:26:51.609 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:51.623 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:51.623 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:51.690 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:51.694 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:52.139 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:52.203 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:26:52.309 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:52.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:52.310 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:26:52.310 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:52.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:52.310 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:26:52.448 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-mirror 2023-01-24T19:26:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:52.840 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:53.719 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:48 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:53.761 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:53.762 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:49 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:55.612 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:26:55.626 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:26:55.626 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:55.627 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-24T19:26:55.627 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:55.627 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:26:55.627 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:26:55.627 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:55.628 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:26:55.628 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:26:55.628 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:26:55.628 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:26:55.629 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 3.5 M 2023-01-24T19:26:55.629 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 16 M 2023-01-24T19:26:55.629 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:26:55.708 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:55.708 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:26:55.709 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:26:55.709 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:55.709 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:26:55.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-nbd 2023-01-24T19:26:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 8.6 MB/s | 3.5 MB 00:00 2023-01-24T19:26:56.029 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:Total 8.6 MB/s | 3.5 MB 00:00 2023-01-24T19:26:56.030 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:26:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:26:56.045 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:26:56.118 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:26:56.123 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:26:56.221 INFO:teuthology.orchestra.run.smithi105.stdout:Updating Subscription Management repositories. 2023-01-24T19:26:56.589 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:26:56.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:57.041 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:53 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:26:57.207 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:58.252 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:58.253 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:53 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:26:58.855 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T19:26:58.856 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:58.857 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2023-01-24T19:26:58.857 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:58.857 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T19:26:58.857 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T19:26:58.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:58.858 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T19:26:58.858 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T19:26:58.858 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T19:26:58.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:26:58.859 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 172 k 2023-01-24T19:26:58.859 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 548 k 2023-01-24T19:26:58.859 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T19:26:59.039 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 946 kB/s | 172 kB 00:00 2023-01-24T19:26:59.040 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:26:59.040 INFO:teuthology.orchestra.run.smithi105.stdout:Total 939 kB/s | 172 kB 00:00 2023-01-24T19:26:59.041 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T19:26:59.052 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T19:26:59.053 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T19:26:59.107 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T19:26:59.111 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T19:26:59.250 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T19:26:59.465 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:26:59.726 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:27:00.169 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:27:00.275 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:00.276 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:27:00.276 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:27:00.276 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:00.277 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:27:00.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-nbd 2023-01-24T19:27:00.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:27:00.776 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:03:56 ago on Tue 24 Jan 2023 07:23:04 PM UTC. 2023-01-24T19:27:00.787 INFO:teuthology.orchestra.run.smithi062.stdout:Updating Subscription Management repositories. 2023-01-24T19:27:01.695 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:03:56 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:27:02.768 INFO:teuthology.orchestra.run.smithi105.stdout:Installed products updated. 2023-01-24T19:27:02.862 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:02.863 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T19:27:02.863 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:27:02.864 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:02.864 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T19:27:02.972 DEBUG:teuthology.parallel:result is None 2023-01-24T19:27:03.562 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-24T19:27:03.562 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:27:03.563 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-01-24T19:27:03.563 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:27:03.563 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-24T19:27:03.563 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T19:27:03.564 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:03.564 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-24T19:27:03.564 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-24T19:27:03.564 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-24T19:27:03.564 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:03.565 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 172 k 2023-01-24T19:27:03.565 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 548 k 2023-01-24T19:27:03.565 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-24T19:27:03.730 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2023-01-24T19:27:03.731 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:27:03.731 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.0 MB/s | 172 kB 00:00 2023-01-24T19:27:03.732 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-24T19:27:03.745 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-24T19:27:03.745 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-24T19:27:03.805 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-24T19:27:03.810 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-24T19:27:03.937 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-24T19:27:04.112 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:27:04.398 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:27:05.437 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:27:05.437 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:04:00 ago on Tue 24 Jan 2023 07:23:05 PM UTC. 2023-01-24T19:27:07.357 INFO:teuthology.orchestra.run.smithi062.stdout:Installed products updated. 2023-01-24T19:27:07.465 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:07.466 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-24T19:27:07.466 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:27:07.466 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:07.466 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-24T19:27:07.634 DEBUG:teuthology.parallel:result is None 2023-01-24T19:27:07.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-24T19:27:07.674 INFO:teuthology.orchestra.run.smithi062.stdout:RedHatEnterprise 2023-01-24T19:27:07.675 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:27:07.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:27:07.844 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T19:27:07.864 INFO:teuthology.orchestra.run.smithi062.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T19:27:07.865 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T19:27:07.865 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T19:27:07.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T19:27:07.912 INFO:teuthology.orchestra.run.smithi105.stdout:RedHatEnterprise 2023-01-24T19:27:07.914 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T19:27:07.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:27:08.086 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T19:27:08.108 INFO:teuthology.orchestra.run.smithi105.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T19:27:08.109 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T19:27:08.109 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T19:27:08.110 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T19:27:08.111 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:08.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T19:27:08.136 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:08.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T19:27:08.176 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T19:27:08.177 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:08.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T19:27:08.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T19:27:08.266 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:08.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T19:27:08.294 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T19:27:08.359 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T19:27:08.360 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:08.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T19:27:08.385 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T19:27:08.449 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:08.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T19:27:08.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T19:27:08.541 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T19:27:08.603 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T19:27:08.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2023-01-24T19:27:08.607 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2023-01-24T19:27:08.631 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T19:27:08.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T19:27:08.671 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T19:27:08.695 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T19:27:08.695 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T19:27:08.735 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T19:27:08.776 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T19:27:08.777 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T19:27:08.777 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135655', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/cache}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135655', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi062.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBTL2Y/PhS5EBGYHttlV9kpZvxPcjdAKTc/toHFeYydBHuV/RPQjTKn09plQ+Sq3Xgm2pdBKpe8sQFT5yQaEzhnKz9kNKCh8EiDQGj67Khw6lZ4U6RcwxdjXiAcfEaRBGatDaBHmrSryTK7259LXCx0lT266f03BoyVcuw3M00HzifXGL5bXdw8dLvZjVSSgkUvuUVZqSEmd9nIF6yBGaR4NPQhTjvnD9v6en9g1RBzMH1xgL4RIegNNBtkanG3pXo8oYKltKfS7GtyWSsgTcPvlNnzOA+RVijt2miZkUnVBN/b8HzHibKUMqHQSznKEI9/PJV2+CotOCGWUQRelO1JpM+uNmx9CWDH7rHgB/JwQDn4+kZ9aIC2N0aJ6lALIehYJoz55G0GEpTxBwMxeQxPYwyU1CFLkbYrLRyNHfofwJBtBt7loiXMxfqmUCj5glz5QLzLy+I3Nl2mIPbxGJGfGkbL4mBBuOJRGgVKmbfYPsAKtWq4FquYMhRsVq53sM=', 'smithi105.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDbWJ9eq7e+26VETaENGgNVnn1rndNRtDKezG1HU7oVihEx6Y4cn7tNlK7WoqZxXFT3JQAUGoiU+K+Cj8SmtcJgmxXXY2zVKoqNgK2yPRfS2bTFaoAaZkCnnC/hs1gcxvZ70LYLtZikUk9F42s8c3G8G4JqVifdrVWPL5HS3yope/eJ9cTkh+7UUnM/yYo8wc9gBvPqU7443qLsuosDUW+HCF+leTR8lo83AvCZRPv+sakL2oP/zW4yq/qIr8swMSvQSVgAn3J7Ww9nBykyRIp2MozOBwSxr/Fb02YJK7WTSSuVVa7E3Q4R+G+IMziajjSGUeN0o/02FjwKWxaQVscUXooORlvC27U6lHxS9o8D6F5pCL0wdZAMjVGFTChl75SAFrVF2nYR9S+7TKpmFNa+cB5eTyAu+E7fcx9ndWV2Juig/vrhOXP2uYhK+TS5i5eJDOdjyOAugoHT88QP1lzXGLLAV/S5nKjY2c+9lxNqGm77Wv/SkccokUkjz6hbOQs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache min_read_recency_for_promote 0', 'sudo ceph osd pool set cache min_write_recency_for_promote 0']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T19:27:08.778 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T19:27:08.799 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T19:27:08.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T19:27:08.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T19:27:08.877 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:08.878 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T19:27:08.929 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T19:27:08.929 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-01-24T19:27:08.983 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T19:27:08.984 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:08.984 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 203920 Links: 1 2023-01-24T19:27:08.984 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:08.984 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:08.985 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-24 19:26:56.969716791 +0000 2023-01-24T19:27:08.985 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-24 19:22:21.556613115 +0000 2023-01-24T19:27:08.985 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-24 19:22:21.556613115 +0000 2023-01-24T19:27:08.985 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-24T19:27:08.985 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T19:27:09.046 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-24T19:27:09.047 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-24T19:27:09.047 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000216155 s, 2.4 MB/s 2023-01-24T19:27:09.048 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T19:27:09.104 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-01-24T19:27:09.159 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T19:27:09.159 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:09.159 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 203960 Links: 1 2023-01-24T19:27:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-24 19:26:56.969716791 +0000 2023-01-24T19:27:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-24 19:22:22.076605155 +0000 2023-01-24T19:27:09.160 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-24 19:22:22.076605155 +0000 2023-01-24T19:27:09.161 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-24T19:27:09.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T19:27:09.221 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-24T19:27:09.222 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-24T19:27:09.222 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000215365 s, 2.4 MB/s 2023-01-24T19:27:09.223 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T19:27:09.277 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-01-24T19:27:09.332 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T19:27:09.332 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:09.332 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 202322 Links: 1 2023-01-24T19:27:09.333 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:09.333 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:09.333 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-24 19:26:56.969716791 +0000 2023-01-24T19:27:09.333 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-24 19:22:22.598597165 +0000 2023-01-24T19:27:09.333 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-24 19:22:22.598597165 +0000 2023-01-24T19:27:09.334 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-24T19:27:09.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T19:27:09.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-24T19:27:09.395 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-24T19:27:09.396 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000206245 s, 2.5 MB/s 2023-01-24T19:27:09.397 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T19:27:09.452 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-01-24T19:27:09.507 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T19:27:09.507 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:09.508 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 206117 Links: 1 2023-01-24T19:27:09.508 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:09.508 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:09.508 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-24 19:26:56.969716791 +0000 2023-01-24T19:27:09.509 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-24 19:22:23.108589358 +0000 2023-01-24T19:27:09.509 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-24 19:22:23.108589358 +0000 2023-01-24T19:27:09.509 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-24T19:27:09.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T19:27:09.570 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-24T19:27:09.571 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-24T19:27:09.571 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000206702 s, 2.5 MB/s 2023-01-24T19:27:09.572 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T19:27:09.628 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:09.628 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:09.628 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T19:27:09.646 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T19:27:09.646 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2023-01-24T19:27:09.702 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T19:27:09.703 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:09.703 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 199648 Links: 1 2023-01-24T19:27:09.703 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:09.703 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:09.703 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 19:26:52.512859750 +0000 2023-01-24T19:27:09.704 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 19:22:08.387467929 +0000 2023-01-24T19:27:09.704 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 19:22:08.387467929 +0000 2023-01-24T19:27:09.704 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T19:27:09.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T19:27:09.768 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T19:27:09.769 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T19:27:09.769 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000256547 s, 2.0 MB/s 2023-01-24T19:27:09.770 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T19:27:09.827 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2023-01-24T19:27:09.882 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T19:27:09.883 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:09.883 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 202206 Links: 1 2023-01-24T19:27:09.883 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:09.883 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:09.883 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 19:26:52.512859750 +0000 2023-01-24T19:27:09.884 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 19:22:08.842458974 +0000 2023-01-24T19:27:09.884 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 19:22:08.842458974 +0000 2023-01-24T19:27:09.884 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T19:27:09.884 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T19:27:09.946 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T19:27:09.947 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T19:27:09.947 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000233604 s, 2.2 MB/s 2023-01-24T19:27:09.948 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T19:27:10.004 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2023-01-24T19:27:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T19:27:10.059 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:10.059 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 206905 Links: 1 2023-01-24T19:27:10.060 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:10.060 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:10.060 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 19:26:52.512859750 +0000 2023-01-24T19:27:10.060 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 19:22:09.371448564 +0000 2023-01-24T19:27:10.060 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 19:22:09.371448564 +0000 2023-01-24T19:27:10.061 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T19:27:10.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T19:27:10.121 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T19:27:10.122 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T19:27:10.122 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000234685 s, 2.2 MB/s 2023-01-24T19:27:10.123 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T19:27:10.178 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2023-01-24T19:27:10.233 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T19:27:10.234 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:27:10.234 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 198532 Links: 1 2023-01-24T19:27:10.234 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:27:10.234 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:27:10.235 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 19:26:52.512859750 +0000 2023-01-24T19:27:10.235 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 19:22:09.842439295 +0000 2023-01-24T19:27:10.235 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 19:22:09.842439295 +0000 2023-01-24T19:27:10.235 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T19:27:10.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T19:27:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T19:27:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T19:27:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000189837 s, 2.7 MB/s 2023-01-24T19:27:10.298 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T19:27:10.353 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:10.353 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T19:27:10.353 INFO:tasks.ceph:Generating config... 2023-01-24T19:27:10.355 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T19:27:10.355 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2023-01-24T19:27:10.356 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2023-01-24T19:27:10.356 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T19:27:10.356 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T19:27:10.357 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T19:27:10.357 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T19:27:10.357 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T19:27:10.357 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T19:27:10.358 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T19:27:10.358 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T19:27:10.358 INFO:tasks.ceph:[osd] filestore odsync write = True 2023-01-24T19:27:10.358 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T19:27:10.359 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T19:27:10.359 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T19:27:10.359 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2023-01-24T19:27:10.359 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-24T19:27:10.360 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T19:27:10.360 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T19:27:10.360 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T19:27:10.360 INFO:tasks.ceph:[osd] osd max backfills = 2 2023-01-24T19:27:10.361 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-24T19:27:10.361 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-01-24T19:27:10.361 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T19:27:10.361 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T19:27:10.362 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T19:27:10.362 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T19:27:10.362 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-24T19:27:10.363 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-24T19:27:10.363 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T19:27:10.363 INFO:tasks.ceph:[osd] osd snap trim sleep = 0.5 2023-01-24T19:27:10.363 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-24T19:27:10.364 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-24T19:27:10.364 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-24T19:27:10.364 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-24T19:27:10.364 INFO:tasks.ceph:[global] ms inject delay max = 1 2023-01-24T19:27:10.365 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2023-01-24T19:27:10.365 INFO:tasks.ceph:[global] ms inject delay type = osd 2023-01-24T19:27:10.365 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2023-01-24T19:27:10.365 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-24T19:27:10.366 INFO:tasks.ceph:[global] ms type = async 2023-01-24T19:27:10.366 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-24T19:27:10.366 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-24T19:27:10.366 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-24T19:27:10.367 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-24T19:27:10.367 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-24T19:27:10.367 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T19:27:10.367 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T19:27:10.368 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T19:27:10.368 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T19:27:10.368 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T19:27:10.368 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-24T19:27:10.369 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-24T19:27:10.369 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-24T19:27:10.369 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-24T19:27:10.369 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T19:27:10.370 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-24T19:27:10.370 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T19:27:10.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T19:27:10.418 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T19:27:10.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T19:27:10.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:27:10.526 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.62'), ('mon.c', '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]'), ('mon.b', '172.21.15.105')] 2023-01-24T19:27:10.526 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790],172.21.15.105', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T19:27:10.527 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:10.528 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T19:27:10.580 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.62 --addv c '[v2:172.21.15.62:3301,v1:172.21.15.62:6790]' --add b 172.21.15.105 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:10.662 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:10.663 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: generated fsid f98da7d2-71a8-4fa9-8756-fa9940165f36 2023-01-24T19:27:10.663 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = octopus 2023-01-24T19:27:10.663 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 0 2023-01-24T19:27:10.663 INFO:teuthology.orchestra.run.smithi062.stdout:fsid f98da7d2-71a8-4fa9-8756-fa9940165f36 2023-01-24T19:27:10.664 INFO:teuthology.orchestra.run.smithi062.stdout:last_changed 2023-01-24T19:27:10.662597+0000 2023-01-24T19:27:10.664 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-01-24T19:27:10.662597+0000 2023-01-24T19:27:10.664 INFO:teuthology.orchestra.run.smithi062.stdout:min_mon_release 15 (octopus) 2023-01-24T19:27:10.664 INFO:teuthology.orchestra.run.smithi062.stdout:election_strategy: 3 2023-01-24T19:27:10.665 INFO:teuthology.orchestra.run.smithi062.stdout:0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2023-01-24T19:27:10.665 INFO:teuthology.orchestra.run.smithi062.stdout:1: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2023-01-24T19:27:10.665 INFO:teuthology.orchestra.run.smithi062.stdout:2: [v2:172.21.15.62:3301/0,v1:172.21.15.62:6790/0] mon.c 2023-01-24T19:27:10.665 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T19:27:10.667 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T19:27:10.721 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f98da7d2-71a8-4fa9-8756-fa9940165f36... 2023-01-24T19:27:10.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T19:27:10.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T19:27:10.804 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2023-01-24T19:27:10.804 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2023-01-24T19:27:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T19:27:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2023-01-24T19:27:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2023-01-24T19:27:10.805 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2023-01-24T19:27:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2023-01-24T19:27:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2023-01-24T19:27:10.806 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2023-01-24T19:27:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2023-01-24T19:27:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T19:27:10.807 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2023-01-24T19:27:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T19:27:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T19:27:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.808 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2023-01-24T19:27:10.809 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2023-01-24T19:27:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2023-01-24T19:27:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2023-01-24T19:27:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T19:27:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T19:27:10.810 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2023-01-24T19:27:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T19:27:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2023-01-24T19:27:10.811 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2023-01-24T19:27:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2023-01-24T19:27:10.812 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2023-01-24T19:27:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2023-01-24T19:27:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790],172.21.15.105" 2023-01-24T19:27:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: mon client directed command retry = 5 2023-01-24T19:27:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: mon election default strategy = 3 2023-01-24T19:27:10.813 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject delay max = 1 2023-01-24T19:27:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject delay probability = 0.005 2023-01-24T19:27:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject delay type = osd 2023-01-24T19:27:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject internal delays = 0.002 2023-01-24T19:27:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject socket failures = 2500 2023-01-24T19:27:10.814 INFO:teuthology.orchestra.run.smithi062.stdout: ms type = async 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: osd_max_pg_log_entries = 2 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: osd_min_pg_log_entries = 1 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pg_log_trim_min = 0 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_min_size = 2 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_size = 3 2023-01-24T19:27:10.815 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = f98da7d2-71a8-4fa9-8756-fa9940165f36 2023-01-24T19:27:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.816 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2023-01-24T19:27:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2023-01-24T19:27:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.816 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2023-01-24T19:27:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 120 2023-01-24T19:27:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2023-01-24T19:27:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2023-01-24T19:27:10.817 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2023-01-24T19:27:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2023-01-24T19:27:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2023-01-24T19:27:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.818 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2023-01-24T19:27:10.819 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2023-01-24T19:27:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2023-01-24T19:27:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2023-01-24T19:27:10.820 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2023-01-24T19:27:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2023-01-24T19:27:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: debug monc = 20 2023-01-24T19:27:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: bluefs allocator = hybrid 2023-01-24T19:27:10.821 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = hybrid 2023-01-24T19:27:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2023-01-24T19:27:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2023-01-24T19:27:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore zero block detection = True 2023-01-24T19:27:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 20 2023-01-24T19:27:10.822 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 20 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 10 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: filestore odsync write = True 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T19:27:10.823 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2023-01-24T19:27:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2023-01-24T19:27:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: osd backoff on degraded = True 2023-01-24T19:27:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: osd backoff on peering = True 2023-01-24T19:27:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify cached snaps = True 2023-01-24T19:27:10.824 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify missing on start = True 2023-01-24T19:27:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2023-01-24T19:27:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: osd max backfills = 2 2023-01-24T19:27:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: osd max markdown count = 1000 2023-01-24T19:27:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: osd max scrubs = 3 2023-01-24T19:27:10.825 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock override recovery settings = True 2023-01-24T19:27:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2023-01-24T19:27:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue = debug_random 2023-01-24T19:27:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue cut off = debug_random 2023-01-24T19:27:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub min interval = 60 2023-01-24T19:27:10.826 INFO:teuthology.orchestra.run.smithi062.stdout: osd shutdown pgref assert = True 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: osd snap trim sleep = 0.5 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: osd_max_pg_log_entries = 3000 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: osd_min_pg_log_entries = 3000 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2023-01-24T19:27:10.827 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-24T19:27:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2023-01-24T19:27:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-01-24T19:27:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-01-24T19:27:10.828 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-24T19:27:10.829 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-01-24T19:27:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2023-01-24T19:27:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-01-24T19:27:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2023-01-24T19:27:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2023-01-24T19:27:10.830 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:27:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2023-01-24T19:27:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2023-01-24T19:27:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:27:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T19:27:10.832 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T19:27:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2023-01-24T19:27:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2023-01-24T19:27:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.833 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2023-01-24T19:27:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: mon min osdmap epochs = 50 2023-01-24T19:27:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune interval = 2 2023-01-24T19:27:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune min = 15 2023-01-24T19:27:10.834 INFO:teuthology.orchestra.run.smithi062.stdout: mon osdmap full prune txsize = 2 2023-01-24T19:27:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: mon scrub interval = 300 2023-01-24T19:27:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: paxos service trim min = 10 2023-01-24T19:27:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:10.835 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2023-01-24T19:27:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2023-01-24T19:27:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2023-01-24T19:27:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2023-01-24T19:27:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T19:27:10.836 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T19:27:10.836 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2023-01-24T19:27:10.837 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2023-01-24T19:27:10.837 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2023-01-24T19:27:10.838 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-01-24T19:27:10.839 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-01-24T19:27:10.839 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T19:27:10.839 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-01-24T19:27:10.839 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.839 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-01-24T19:27:10.840 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.840 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-01-24T19:27:10.840 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.840 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-01-24T19:27:10.840 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-01-24T19:27:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-01-24T19:27:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-01-24T19:27:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.841 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.842 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-01-24T19:27:10.843 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.843 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-01-24T19:27:10.843 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-01-24T19:27:10.843 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-01-24T19:27:10.843 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-01-24T19:27:10.844 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-01-24T19:27:10.844 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-01-24T19:27:10.844 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-01-24T19:27:10.844 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T19:27:10.844 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T19:27:10.845 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-01-24T19:27:10.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.845 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T19:27:10.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.845 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-01-24T19:27:10.846 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-01-24T19:27:10.847 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790],172.21.15.105" 2023-01-24T19:27:10.847 INFO:teuthology.orchestra.run.smithi105.stdout: mon client directed command retry = 5 2023-01-24T19:27:10.847 INFO:teuthology.orchestra.run.smithi105.stdout: mon election default strategy = 3 2023-01-24T19:27:10.847 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay max = 1 2023-01-24T19:27:10.847 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay probability = 0.005 2023-01-24T19:27:10.848 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay type = osd 2023-01-24T19:27:10.848 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject internal delays = 0.002 2023-01-24T19:27:10.848 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject socket failures = 2500 2023-01-24T19:27:10.848 INFO:teuthology.orchestra.run.smithi105.stdout: ms type = async 2023-01-24T19:27:10.848 INFO:teuthology.orchestra.run.smithi105.stdout: osd_max_pg_log_entries = 2 2023-01-24T19:27:10.849 INFO:teuthology.orchestra.run.smithi105.stdout: osd_min_pg_log_entries = 1 2023-01-24T19:27:10.849 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pg_log_trim_min = 0 2023-01-24T19:27:10.849 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_min_size = 2 2023-01-24T19:27:10.849 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_size = 3 2023-01-24T19:27:10.849 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = f98da7d2-71a8-4fa9-8756-fa9940165f36 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-01-24T19:27:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 120 2023-01-24T19:27:10.851 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.851 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-01-24T19:27:10.851 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-01-24T19:27:10.851 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.851 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-01-24T19:27:10.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-01-24T19:27:10.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-01-24T19:27:10.852 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-01-24T19:27:10.852 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.853 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-01-24T19:27:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-01-24T19:27:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-01-24T19:27:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.854 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-01-24T19:27:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-01-24T19:27:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: debug monc = 20 2023-01-24T19:27:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: bluefs allocator = hybrid 2023-01-24T19:27:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = hybrid 2023-01-24T19:27:10.855 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2023-01-24T19:27:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2023-01-24T19:27:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore zero block detection = True 2023-01-24T19:27:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2023-01-24T19:27:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2023-01-24T19:27:10.856 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: filestore odsync write = True 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2023-01-24T19:27:10.857 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2023-01-24T19:27:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd backoff on degraded = True 2023-01-24T19:27:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd backoff on peering = True 2023-01-24T19:27:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify cached snaps = True 2023-01-24T19:27:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify missing on start = True 2023-01-24T19:27:10.858 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2023-01-24T19:27:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd max backfills = 2 2023-01-24T19:27:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd max markdown count = 1000 2023-01-24T19:27:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd max scrubs = 3 2023-01-24T19:27:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock override recovery settings = True 2023-01-24T19:27:10.859 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd op queue = debug_random 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd op queue cut off = debug_random 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub min interval = 60 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd shutdown pgref assert = True 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd snap trim sleep = 0.5 2023-01-24T19:27:10.860 INFO:teuthology.orchestra.run.smithi105.stdout: osd_max_pg_log_entries = 3000 2023-01-24T19:27:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: osd_min_pg_log_entries = 3000 2023-01-24T19:27:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.861 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-01-24T19:27:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T19:27:10.861 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-01-24T19:27:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-01-24T19:27:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-01-24T19:27:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:27:10.862 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-01-24T19:27:10.863 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-01-24T19:27:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-01-24T19:27:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-01-24T19:27:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-01-24T19:27:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:27:10.864 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-01-24T19:27:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-01-24T19:27:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:27:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T19:27:10.865 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T19:27:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T19:27:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T19:27:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T19:27:10.866 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-01-24T19:27:10.867 INFO:teuthology.orchestra.run.smithi105.stdout: mon min osdmap epochs = 50 2023-01-24T19:27:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune interval = 2 2023-01-24T19:27:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune min = 15 2023-01-24T19:27:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune txsize = 2 2023-01-24T19:27:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: mon scrub interval = 300 2023-01-24T19:27:10.868 INFO:teuthology.orchestra.run.smithi105.stdout: paxos service trim min = 10 2023-01-24T19:27:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T19:27:10.869 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-01-24T19:27:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-01-24T19:27:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-01-24T19:27:10.869 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-01-24T19:27:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T19:27:10.870 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T19:27:10.870 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-01-24T19:27:10.870 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2023-01-24T19:27:10.870 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2023-01-24T19:27:10.871 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T19:27:10.871 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T19:27:10.929 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T19:27:10.930 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:10.930 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T19:27:10.945 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:10.945 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T19:27:11.000 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:27:11.000 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:11.000 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T19:27:11.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:27:11.073 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:11.073 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:11.128 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:27:11.128 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:11.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T19:27:11.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:27:11.163 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:11.164 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:11.219 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T19:27:11.219 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T19:27:11.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T19:27:11.268 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T19:27:11.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T19:27:11.333 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T19:27:11.336 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T19:27:11.337 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T19:27:11.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T19:27:11.376 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T19:27:11.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T19:27:11.425 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T19:27:11.437 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T19:27:11.437 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T19:27:11.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T19:27:11.462 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:11.462 INFO:tasks.ceph:role: osd.0 2023-01-24T19:27:11.462 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:27:11.463 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T19:27:11.533 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:11.533 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:11.534 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:11.535 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:11.535 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:11.893 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-24T19:27:11.895 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-24T19:27:11.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T19:27:12.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T19:27:12.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T19:27:12.112 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:12.112 INFO:tasks.ceph:role: osd.1 2023-01-24T19:27:12.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:27:12.113 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T19:27:12.184 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:12.184 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:12.184 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:12.185 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-01-24T19:27:12.185 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:12.185 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:12.185 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:12.185 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:12.186 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:12.186 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:12.547 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-24T19:27:12.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-24T19:27:12.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T19:27:12.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T19:27:12.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T19:27:12.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:12.728 INFO:tasks.ceph:role: osd.2 2023-01-24T19:27:12.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:27:12.729 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T19:27:12.800 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:12.801 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:12.801 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:12.801 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-01-24T19:27:12.801 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:12.802 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:12.802 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:12.802 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:12.802 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:12.802 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:13.169 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-24T19:27:13.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-24T19:27:13.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T19:27:13.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T19:27:13.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T19:27:13.373 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:13.373 INFO:tasks.ceph:role: osd.3 2023-01-24T19:27:13.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-24T19:27:13.374 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T19:27:13.445 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:13.445 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:13.445 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:13.445 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 2023-01-24T19:27:13.446 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:13.446 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:13.446 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:13.446 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:13.446 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:13.447 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:13.819 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-24T19:27:13.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-24T19:27:13.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T19:27:13.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T19:27:13.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:14.032 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.030+0000 7fec18222540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T19:27:14.033 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.031+0000 7fec18222540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T19:27:14.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.031+0000 7fec18222540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:27:14.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.031+0000 7fec18222540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:27:14.034 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.031+0000 7fec18222540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:27:14.036 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:14.035+0000 7fec18222540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T19:27:16.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T19:27:16.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.054+0000 7f7a72d71540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T19:27:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.055+0000 7f7a72d71540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T19:27:17.056 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.055+0000 7f7a72d71540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:27:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.055+0000 7f7a72d71540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:27:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.055+0000 7f7a72d71540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:27:17.057 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:17.055+0000 7f7a72d71540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T19:27:19.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T19:27:19.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T19:27:20.065 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T19:27:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:27:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:27:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:27:20.066 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:20.064+0000 7fc49f6c2540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T19:27:22.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T19:27:22.951 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T19:27:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T19:27:23.037 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:27:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:27:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:27:23.038 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:27:23.036+0000 7faec020d540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T19:27:25.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T19:27:25.920 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T19:27:25.950 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:25.950 INFO:tasks.ceph:role: osd.4 2023-01-24T19:27:25.950 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:27:25.950 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T19:27:26.026 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:26.026 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:26.026 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:26.026 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 2023-01-24T19:27:26.027 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:26.027 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:26.027 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:26.027 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:26.027 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:26.028 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:26.392 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T19:27:26.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T19:27:26.394 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T19:27:26.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T19:27:26.554 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T19:27:26.621 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:26.621 INFO:tasks.ceph:role: osd.5 2023-01-24T19:27:26.622 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:27:26.622 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T19:27:26.693 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 2023-01-24T19:27:26.694 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:26.694 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:26.695 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:26.695 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:26.695 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:26.695 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:27.059 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T19:27:27.061 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T19:27:27.062 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T19:27:27.154 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T19:27:27.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T19:27:27.248 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:27.248 INFO:tasks.ceph:role: osd.6 2023-01-24T19:27:27.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:27:27.249 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T19:27:27.320 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:27.320 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:27.321 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:27.321 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 2023-01-24T19:27:27.321 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:27.321 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:27.322 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:27.322 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:27.322 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:27.322 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:27.695 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T19:27:27.697 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T19:27:27.698 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T19:27:27.804 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T19:27:27.834 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T19:27:27.900 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:27:27.901 INFO:tasks.ceph:role: osd.7 2023-01-24T19:27:27.901 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T19:27:27.902 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T19:27:27.974 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:27:27.975 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:27:27.975 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:27:27.975 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 2023-01-24T19:27:27.976 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:27:27.976 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:27:27.976 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:27:27.977 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:27:27.977 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:27:27.978 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:27:28.345 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T19:27:28.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T19:27:28.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T19:27:28.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T19:27:28.489 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:28.573 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.572+0000 7f76ddeba540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T19:27:28.575 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.574+0000 7f76ddeba540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T19:27:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.574+0000 7f76ddeba540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:27:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.574+0000 7f76ddeba540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:27:28.576 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.574+0000 7f76ddeba540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:27:28.584 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:28.583+0000 7f76ddeba540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T19:27:31.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T19:27:31.470 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:31.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.550+0000 7fa9e8ca8540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T19:27:31.552 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.551+0000 7fa9e8ca8540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T19:27:31.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.551+0000 7fa9e8ca8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:27:31.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.551+0000 7fa9e8ca8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:27:31.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.551+0000 7fa9e8ca8540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:27:31.553 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:31.551+0000 7fa9e8ca8540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T19:27:34.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T19:27:34.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.562+0000 7f3e888e6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T19:27:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.562+0000 7f3e888e6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T19:27:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.562+0000 7f3e888e6540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:27:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.562+0000 7f3e888e6540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:27:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.562+0000 7f3e888e6540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:27:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:34.563+0000 7f3e888e6540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T19:27:37.416 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T19:27:37.441 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:37.521 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.519+0000 7f0c98f89540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T19:27:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.519+0000 7f0c98f89540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T19:27:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.519+0000 7f0c98f89540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:27:37.522 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.519+0000 7f0c98f89540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:27:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.519+0000 7f0c98f89540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:27:37.523 INFO:teuthology.orchestra.run.smithi105.stderr:2023-01-24T19:27:37.520+0000 7f0c98f89540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T19:27:40.369 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T19:27:40.435 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T19:27:40.435 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:40.436 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T19:27:40.465 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:40.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T19:27:40.534 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:40.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T19:27:40.602 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:40.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T19:27:40.669 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:40.670 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T19:27:40.736 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:40.736 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T19:27:40.764 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:40.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T19:27:40.831 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:40.832 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T19:27:40.897 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:40.898 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T19:27:40.963 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:40.964 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T19:27:41.028 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:41.028 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T19:27:41.045 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:41.046 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T19:27:41.082 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T19:27:41.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T19:27:41.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T19:27:41.126 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2023-01-24T19:27:41.126 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPMdBjtZP2DxAAs8WJY2IDmhXkxvP7nmdPHA== 2023-01-24T19:27:41.126 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2023-01-24T19:27:41.126 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQASMdBjGpvhARAAku3tNraNgyjRS3XyvySC9Q== 2023-01-24T19:27:41.127 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2023-01-24T19:27:41.127 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAVMdBjQ6pOAxAA6/10rOWbEMHTCo+ljT+3qw== 2023-01-24T19:27:41.127 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2023-01-24T19:27:41.127 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAYMdBj1qjaAxAA+7gWKnnZxk8ggJwNlhY4fA== 2023-01-24T19:27:41.128 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2023-01-24T19:27:41.128 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAbMdBjwWEwAhAAbyKjEvlwxmcArLtbezvNFg== 2023-01-24T19:27:41.128 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2023-01-24T19:27:41.128 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPMdBjF1bkExAAJoMRC/Tg8tWfxYVc3ASwrQ== 2023-01-24T19:27:41.128 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2023-01-24T19:27:41.129 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAgMdBjSp8yIhAAwSCOfv/22+1RXT9FTMbT+A== 2023-01-24T19:27:41.129 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2023-01-24T19:27:41.129 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAjMdBjotXiIBAAnnLjUu+UHK9UT1rMmk1kYA== 2023-01-24T19:27:41.129 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2023-01-24T19:27:41.129 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAmMdBjj5KTIRAA3udtP78j7bHpq6uFbEoiFQ== 2023-01-24T19:27:41.130 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2023-01-24T19:27:41.130 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQApMdBjt4ISHxAAl/5aSABl/QU7gj/GDCQezA== 2023-01-24T19:27:41.130 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2023-01-24T19:27:41.130 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPMdBjfCtoFhAAw4L8a9SoW8dvVZYsFqEtfw== 2023-01-24T19:27:41.130 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2023-01-24T19:27:41.131 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAPMdBjbVJWGRAAHLQumnHggEb7tcHhYwOZlA== 2023-01-24T19:27:41.145 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2023-01-24T19:27:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAPMdBjtZP2DxAAs8WJY2IDmhXkxvP7nmdPHA== 2023-01-24T19:27:41.146 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2023-01-24T19:27:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQASMdBjGpvhARAAku3tNraNgyjRS3XyvySC9Q== 2023-01-24T19:27:41.146 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2023-01-24T19:27:41.146 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAVMdBjQ6pOAxAA6/10rOWbEMHTCo+ljT+3qw== 2023-01-24T19:27:41.147 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2023-01-24T19:27:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAYMdBj1qjaAxAA+7gWKnnZxk8ggJwNlhY4fA== 2023-01-24T19:27:41.147 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2023-01-24T19:27:41.147 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAbMdBjwWEwAhAAbyKjEvlwxmcArLtbezvNFg== 2023-01-24T19:27:41.147 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2023-01-24T19:27:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAPMdBjF1bkExAAJoMRC/Tg8tWfxYVc3ASwrQ== 2023-01-24T19:27:41.148 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2023-01-24T19:27:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAgMdBjSp8yIhAAwSCOfv/22+1RXT9FTMbT+A== 2023-01-24T19:27:41.148 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2023-01-24T19:27:41.148 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAjMdBjotXiIBAAnnLjUu+UHK9UT1rMmk1kYA== 2023-01-24T19:27:41.149 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2023-01-24T19:27:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAmMdBjj5KTIRAA3udtP78j7bHpq6uFbEoiFQ== 2023-01-24T19:27:41.149 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2023-01-24T19:27:41.149 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQApMdBjt4ISHxAAl/5aSABl/QU7gj/GDCQezA== 2023-01-24T19:27:41.149 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-01-24T19:27:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAPMdBjfCtoFhAAw4L8a9SoW8dvVZYsFqEtfw== 2023-01-24T19:27:41.150 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2023-01-24T19:27:41.150 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAPMdBjbVJWGRAAHLQumnHggEb7tcHhYwOZlA== 2023-01-24T19:27:41.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:27:41.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:27:41.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.340 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:27:41.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:27:41.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:27:41.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:27:41.675 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:27:41.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:27:41.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:27:41.775 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T19:27:41.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T19:27:41.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:27:41.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T19:27:41.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T19:27:41.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:27:42.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T19:27:42.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T19:27:42.120 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:27:42.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T19:27:42.233 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:42.235 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:27:42.286 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T19:27:42.287 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T19:27:42.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T19:27:42.289 INFO:tasks.ceph.mon.a:Started 2023-01-24T19:27:42.290 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T19:27:42.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T19:27:42.292 INFO:tasks.ceph.mon.c:Started 2023-01-24T19:27:42.293 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T19:27:42.293 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T19:27:42.329 INFO:tasks.ceph.mon.b:Started 2023-01-24T19:27:42.329 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T19:27:42.330 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T19:27:42.332 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T19:27:42.334 INFO:tasks.ceph.mgr.y:Started 2023-01-24T19:27:42.334 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T19:27:42.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T19:27:42.336 INFO:tasks.ceph.mgr.x:Started 2023-01-24T19:27:42.337 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T19:27:42.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T19:27:48.744 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2023-01-24T19:27:48.756 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T19:27:48.757 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:48.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T19:27:48.782 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:48.782 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T19:27:48.846 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:48.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T19:27:48.911 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-24T19:27:48.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T19:27:48.975 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:48.976 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T19:27:49.000 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:49.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T19:27:49.063 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:49.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T19:27:49.128 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T19:27:49.128 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T19:27:49.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 2e461ba5-43f1-46b1-b762-74211e41888b 0 2023-01-24T19:27:49.440 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T19:27:49.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new ccfe4647-f69d-443a-a543-43082d69dae1 1 2023-01-24T19:27:49.665 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-01-24T19:27:49.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 5ece0928-7625-452c-9bb2-b4e7d8c66bca 2 2023-01-24T19:27:49.832 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:49.830+0000 7fd5f1915180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T19:27:49.833 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:49.830+0000 7fd5f1915180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T19:27:49.833 INFO:tasks.ceph.mgr.y.smithi062.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T19:27:49.833 INFO:tasks.ceph.mgr.y.smithi062.stderr: from .module import Module, StandbyModule 2023-01-24T19:27:49.833 INFO:tasks.ceph.mgr.y.smithi062.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T19:27:49.833 INFO:tasks.ceph.mgr.y.smithi062.stderr: from packaging.version import Version 2023-01-24T19:27:49.834 INFO:tasks.ceph.mgr.y.smithi062.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T19:27:49.834 INFO:tasks.ceph.mgr.y.smithi062.stderr: 2023-01-24T19:27:49.834 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:49.830+0000 7fd5f1915180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T19:27:49.834 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:49.830+0000 7fd5f1915180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T19:27:49.891 INFO:teuthology.orchestra.run.smithi105.stdout:2 2023-01-24T19:27:49.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 977e3c9e-7585-45de-a899-41d3e8690a39 3 2023-01-24T19:27:49.969 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:49.968+0000 7fd5f1915180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T19:27:50.014 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.013+0000 7f481c5f1180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T19:27:50.014 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.013+0000 7f481c5f1180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T19:27:50.015 INFO:tasks.ceph.mgr.x.smithi105.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T19:27:50.016 INFO:tasks.ceph.mgr.x.smithi105.stderr: from .module import Module, StandbyModule 2023-01-24T19:27:50.016 INFO:tasks.ceph.mgr.x.smithi105.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T19:27:50.016 INFO:tasks.ceph.mgr.x.smithi105.stderr: from packaging.version import Version 2023-01-24T19:27:50.016 INFO:tasks.ceph.mgr.x.smithi105.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T19:27:50.017 INFO:tasks.ceph.mgr.x.smithi105.stderr: 2023-01-24T19:27:50.017 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.013+0000 7f481c5f1180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T19:27:50.017 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.013+0000 7f481c5f1180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T19:27:50.047 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.046+0000 7fd5f1915180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T19:27:50.128 INFO:teuthology.orchestra.run.smithi105.stdout:3 2023-01-24T19:27:50.140 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new acff091a-d719-45ee-934a-e006744a6d17 4 2023-01-24T19:27:50.157 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.156+0000 7f481c5f1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T19:27:50.234 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.233+0000 7f481c5f1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T19:27:50.238 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.236+0000 7fd5f1915180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:27:50.363 INFO:teuthology.orchestra.run.smithi105.stdout:4 2023-01-24T19:27:50.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new ade74c56-2106-4a4f-b9de-0f779c624290 5 2023-01-24T19:27:50.416 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.414+0000 7fd5f1915180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T19:27:50.425 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.424+0000 7f481c5f1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:27:50.496 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.494+0000 7fd5f1915180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T19:27:50.602 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.601+0000 7f481c5f1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T19:27:50.626 INFO:teuthology.orchestra.run.smithi105.stdout:5 2023-01-24T19:27:50.636 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.635+0000 7fd5f1915180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T19:27:50.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new f0859447-fa75-46cb-ad6e-640fdf2ba9d3 6 2023-01-24T19:27:50.680 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.679+0000 7f481c5f1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T19:27:50.735 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:50.733+0000 7fd5f1915180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T19:27:50.820 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.819+0000 7f481c5f1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T19:27:50.891 INFO:teuthology.orchestra.run.smithi105.stdout:6 2023-01-24T19:27:50.904 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 02f33469-e62e-4f3a-94e1-537535e69e2a 7 2023-01-24T19:27:50.916 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:50.915+0000 7f481c5f1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T19:27:51.118 INFO:teuthology.orchestra.run.smithi105.stdout:7 2023-01-24T19:27:51.130 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T19:27:51.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T19:27:51.133 INFO:tasks.ceph.osd.0:Started 2023-01-24T19:27:51.134 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T19:27:51.134 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T19:27:51.136 INFO:tasks.ceph.osd.1:Started 2023-01-24T19:27:51.137 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T19:27:51.137 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T19:27:51.139 INFO:tasks.ceph.osd.2:Started 2023-01-24T19:27:51.140 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T19:27:51.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T19:27:51.142 INFO:tasks.ceph.osd.3:Started 2023-01-24T19:27:51.142 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T19:27:51.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T19:27:51.172 INFO:tasks.ceph.osd.4:Started 2023-01-24T19:27:51.173 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T19:27:51.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T19:27:51.175 INFO:tasks.ceph.osd.5:Started 2023-01-24T19:27:51.175 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T19:27:51.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T19:27:51.178 INFO:tasks.ceph.osd.6:Started 2023-01-24T19:27:51.178 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T19:27:51.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T19:27:51.181 INFO:tasks.ceph.osd.7:Started 2023-01-24T19:27:51.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:27:51.297 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:51.295+0000 7fd5f1915180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T19:27:51.379 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:51.377+0000 7fd5f1915180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T19:27:51.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:27:51.393 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"fsid":"f98da7d2-71a8-4fa9-8756-fa9940165f36","created":"2023-01-24T19:27:48.609675+0000","modified":"2023-01-24T19:27:51.112886+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T19:27:51.112886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2e461ba5-43f1-46b1-b762-74211e41888b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ccfe4647-f69d-443a-a543-43082d69dae1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ece0928-7625-452c-9bb2-b4e7d8c66bca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"977e3c9e-7585-45de-a899-41d3e8690a39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"acff091a-d719-45ee-934a-e006744a6d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ade74c56-2106-4a4f-b9de-0f779c624290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0859447-fa75-46cb-ad6e-640fdf2ba9d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"02f33469-e62e-4f3a-94e1-537535e69e2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:27:51.405 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T19:27:51.406 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T19:27:51.443 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:51.442+0000 7f481c5f1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T19:27:51.460 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:51.458+0000 7fd5f1915180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T19:27:51.518 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:51.517+0000 7f481c5f1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T19:27:51.595 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:51.594+0000 7f481c5f1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T19:27:51.738 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-24T19:27:51.736+0000 7f7a88887540 -1 Falling back to public interface 2023-01-24T19:27:51.745 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-24T19:27:51.743+0000 7f59d3741540 -1 Falling back to public interface 2023-01-24T19:27:51.754 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-24T19:27:51.752+0000 7f019a781540 -1 Falling back to public interface 2023-01-24T19:27:51.766 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-01-24T19:27:51.765+0000 7f621e4d1540 -1 Falling back to public interface 2023-01-24T19:27:51.768 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-24T19:27:51.766+0000 7eff63b63540 -1 Falling back to public interface 2023-01-24T19:27:51.781 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-01-24T19:27:51.780+0000 7fef50ade540 -1 Falling back to public interface 2023-01-24T19:27:51.787 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-01-24T19:27:51.786+0000 7f9bdb81c540 -1 Falling back to public interface 2023-01-24T19:27:51.798 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-01-24T19:27:51.796+0000 7f5c2af5f540 -1 Falling back to public interface 2023-01-24T19:27:52.077 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:52.075+0000 7fd5f1915180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T19:27:52.158 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:52.156+0000 7f481c5f1180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T19:27:52.212 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T19:27:52.219 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:52.217+0000 7fd5f1915180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:27:52.291 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:52.290+0000 7f481c5f1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:27:52.323 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:52.321+0000 7fd5f1915180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T19:27:52.384 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:52.383+0000 7f481c5f1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T19:27:52.403 INFO:teuthology.misc.health.smithi062.stdout: 2023-01-24T19:27:52.403 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":10,"fsid":"f98da7d2-71a8-4fa9-8756-fa9940165f36","created":"2023-01-24T19:27:48.609675+0000","modified":"2023-01-24T19:27:51.112886+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T19:27:51.112886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2e461ba5-43f1-46b1-b762-74211e41888b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ccfe4647-f69d-443a-a543-43082d69dae1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5ece0928-7625-452c-9bb2-b4e7d8c66bca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"977e3c9e-7585-45de-a899-41d3e8690a39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"acff091a-d719-45ee-934a-e006744a6d17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ade74c56-2106-4a4f-b9de-0f779c624290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f0859447-fa75-46cb-ad6e-640fdf2ba9d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"02f33469-e62e-4f3a-94e1-537535e69e2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:27:52.416 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T19:27:52.580 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:52.579+0000 7fd5f1915180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T19:27:52.625 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:52.624+0000 7f481c5f1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T19:27:52.661 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:52.659+0000 7fd5f1915180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T19:27:52.702 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:52.701+0000 7f481c5f1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T19:27:53.264 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.262+0000 7fd5f1915180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T19:27:53.270 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.269+0000 7f481c5f1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T19:27:53.507 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.506+0000 7f481c5f1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T19:27:53.513 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.511+0000 7fd5f1915180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T19:27:53.635 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.634+0000 7f481c5f1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T19:27:53.651 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.649+0000 7fd5f1915180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T19:27:53.718 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.717+0000 7f481c5f1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T19:27:53.738 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.736+0000 7fd5f1915180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T19:27:53.865 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.863+0000 7f481c5f1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T19:27:53.894 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.892+0000 7fd5f1915180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T19:27:53.937 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:53.935+0000 7f481c5f1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T19:27:53.972 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:53.970+0000 7fd5f1915180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T19:27:54.011 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:54.010+0000 7f481c5f1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T19:27:54.049 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:54.047+0000 7fd5f1915180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T19:27:54.084 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:54.083+0000 7f481c5f1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T19:27:54.084 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:54.083+0000 7f481c5f1180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T19:27:54.128 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:54.126+0000 7fd5f1915180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T19:27:54.128 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-24T19:27:54.126+0000 7fd5f1915180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T19:27:55.111 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:55.110+0000 7f47d5abe700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T19:27:55.115 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T19:27:55.114+0000 7f47d5abe700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T19:27:55.238 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-24T19:27:55.236+0000 7f7a88887540 -1 osd.0 0 log_to_monitors true 2023-01-24T19:27:55.260 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-24T19:27:55.258+0000 7f59d3741540 -1 osd.1 0 log_to_monitors true 2023-01-24T19:27:55.273 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-01-24T19:27:55.272+0000 7f621e4d1540 -1 osd.5 0 log_to_monitors true 2023-01-24T19:27:55.301 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-24T19:27:55.299+0000 7eff63b63540 -1 osd.3 0 log_to_monitors true 2023-01-24T19:27:55.303 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-24T19:27:55.301+0000 7f019a781540 -1 osd.2 0 log_to_monitors true 2023-01-24T19:27:55.326 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-01-24T19:27:55.325+0000 7f5c2af5f540 -1 osd.6 0 log_to_monitors true 2023-01-24T19:27:55.327 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-01-24T19:27:55.326+0000 7fef50ade540 -1 osd.7 0 log_to_monitors true 2023-01-24T19:27:55.335 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-01-24T19:27:55.334+0000 7f9bdb81c540 -1 osd.4 0 log_to_monitors true 2023-01-24T19:27:57.115 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-01-24T19:27:57.114+0000 7f62168f5700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T19:27:57.116 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-24T19:27:57.114+0000 7f59ca362700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T19:27:57.117 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-24T19:27:57.115+0000 7eff5a784700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T19:27:57.118 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-01-24T19:27:57.117+0000 7fef476ff700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T19:27:57.119 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-24T19:27:57.116+0000 7f01913a2700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T19:27:57.119 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-01-24T19:27:57.118+0000 7f9bd243d700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T19:27:57.120 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-24T19:27:57.117+0000 7f7a80cab700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T19:27:57.121 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-01-24T19:27:57.120+0000 7f5c21b80700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T19:27:57.124 INFO:tasks.ceph.osd.4.smithi105.stderr:2023-01-24T19:27:57.123+0000 7f9bcda38700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.127 INFO:tasks.ceph.osd.6.smithi105.stderr:2023-01-24T19:27:57.126+0000 7f5c1d17b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.128 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-24T19:27:57.126+0000 7f018c99d700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.128 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-24T19:27:57.126+0000 7f59c595d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.129 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-24T19:27:57.126+0000 7eff55d7f700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.131 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-24T19:27:57.129+0000 7f7a7aaa3700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.138 INFO:tasks.ceph.osd.7.smithi105.stderr:2023-01-24T19:27:57.137+0000 7fef42cfa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:57.141 INFO:tasks.ceph.osd.5.smithi105.stderr:2023-01-24T19:27:57.140+0000 7f62106ed700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:27:59.222 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T19:27:59.548 INFO:teuthology.misc.health.smithi062.stdout: 2023-01-24T19:27:59.549 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":14,"fsid":"f98da7d2-71a8-4fa9-8756-fa9940165f36","created":"2023-01-24T19:27:48.609675+0000","modified":"2023-01-24T19:27:59.115311+0000","last_up_change":"2023-01-24T19:27:58.113682+0000","last_in_change":"2023-01-24T19:27:51.112886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2e461ba5-43f1-46b1-b762-74211e41888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":82458},{"type":"v1","addr":"172.21.15.62:6801","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":82458},{"type":"v1","addr":"172.21.15.62:6803","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":82458},{"type":"v1","addr":"172.21.15.62:6807","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":82458},{"type":"v1","addr":"172.21.15.62:6805","nonce":82458}]},"public_addr":"172.21.15.62:6801/82458","cluster_addr":"172.21.15.62:6803/82458","heartbeat_back_addr":"172.21.15.62:6807/82458","heartbeat_front_addr":"172.21.15.62:6805/82458","state":["exists","up"]},{"osd":1,"uuid":"ccfe4647-f69d-443a-a543-43082d69dae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":82464},{"type":"v1","addr":"172.21.15.62:6809","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":82464},{"type":"v1","addr":"172.21.15.62:6811","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":82464},{"type":"v1","addr":"172.21.15.62:6815","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":82464},{"type":"v1","addr":"172.21.15.62:6813","nonce":82464}]},"public_addr":"172.21.15.62:6809/82464","cluster_addr":"172.21.15.62:6811/82464","heartbeat_back_addr":"172.21.15.62:6815/82464","heartbeat_front_addr":"172.21.15.62:6813/82464","state":["exists","up"]},{"osd":2,"uuid":"5ece0928-7625-452c-9bb2-b4e7d8c66bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":82483},{"type":"v1","addr":"172.21.15.62:6817","nonce":82483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":82483},{"type":"v1","addr":"172.21.15.62:6819","nonce":82483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":82483},{"type":"v1","addr":"172.21.15.62:6823","nonce":82483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":82483},{"type":"v1","addr":"172.21.15.62:6821","nonce":82483}]},"public_addr":"172.21.15.62:6817/82483","cluster_addr":"172.21.15.62:6819/82483","heartbeat_back_addr":"172.21.15.62:6823/82483","heartbeat_front_addr":"172.21.15.62:6821/82483","state":["exists","up"]},{"osd":3,"uuid":"977e3c9e-7585-45de-a899-41d3e8690a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":82492},{"type":"v1","addr":"172.21.15.62:6825","nonce":82492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":82492},{"type":"v1","addr":"172.21.15.62:6827","nonce":82492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":82492},{"type":"v1","addr":"172.21.15.62:6831","nonce":82492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":82492},{"type":"v1","addr":"172.21.15.62:6829","nonce":82492}]},"public_addr":"172.21.15.62:6825/82492","cluster_addr":"172.21.15.62:6827/82492","heartbeat_back_addr":"172.21.15.62:6831/82492","heartbeat_front_addr":"172.21.15.62:6829/82492","state":["exists","up"]},{"osd":4,"uuid":"acff091a-d719-45ee-934a-e006744a6d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":82421},{"type":"v1","addr":"172.21.15.105:6817","nonce":82421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":82421},{"type":"v1","addr":"172.21.15.105:6819","nonce":82421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":82421},{"type":"v1","addr":"172.21.15.105:6823","nonce":82421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":82421},{"type":"v1","addr":"172.21.15.105:6821","nonce":82421}]},"public_addr":"172.21.15.105:6817/82421","cluster_addr":"172.21.15.105:6819/82421","heartbeat_back_addr":"172.21.15.105:6823/82421","heartbeat_front_addr":"172.21.15.105:6821/82421","state":["exists","up"]},{"osd":5,"uuid":"ade74c56-2106-4a4f-b9de-0f779c624290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":82419},{"type":"v1","addr":"172.21.15.105:6801","nonce":82419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":82419},{"type":"v1","addr":"172.21.15.105:6803","nonce":82419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":82419},{"type":"v1","addr":"172.21.15.105:6807","nonce":82419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":82419},{"type":"v1","addr":"172.21.15.105:6805","nonce":82419}]},"public_addr":"172.21.15.105:6801/82419","cluster_addr":"172.21.15.105:6803/82419","heartbeat_back_addr":"172.21.15.105:6807/82419","heartbeat_front_addr":"172.21.15.105:6805/82419","state":["exists","up"]},{"osd":6,"uuid":"f0859447-fa75-46cb-ad6e-640fdf2ba9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":82436},{"type":"v1","addr":"172.21.15.105:6825","nonce":82436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":82436},{"type":"v1","addr":"172.21.15.105:6827","nonce":82436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":82436},{"type":"v1","addr":"172.21.15.105:6831","nonce":82436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":82436},{"type":"v1","addr":"172.21.15.105:6829","nonce":82436}]},"public_addr":"172.21.15.105:6825/82436","cluster_addr":"172.21.15.105:6827/82436","heartbeat_back_addr":"172.21.15.105:6831/82436","heartbeat_front_addr":"172.21.15.105:6829/82436","state":["exists","up"]},{"osd":7,"uuid":"02f33469-e62e-4f3a-94e1-537535e69e2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":82420},{"type":"v1","addr":"172.21.15.105:6809","nonce":82420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":82420},{"type":"v1","addr":"172.21.15.105:6811","nonce":82420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":82420},{"type":"v1","addr":"172.21.15.105:6815","nonce":82420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":82420},{"type":"v1","addr":"172.21.15.105:6813","nonce":82420}]},"public_addr":"172.21.15.105:6809/82420","cluster_addr":"172.21.15.105:6811/82420","heartbeat_back_addr":"172.21.15.105:6815/82420","heartbeat_front_addr":"172.21.15.105:6813/82420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.264706+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.246881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.319854+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.253673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.298711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.227684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.357601+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.343005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:27:59.561 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T19:27:59.561 INFO:tasks.ceph:Creating RBD pool 2023-01-24T19:27:59.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T19:28:00.335 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'rbd' created 2023-01-24T19:28:00.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T19:28:01.337 INFO:teuthology.orchestra.run.smithi062.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T19:28:01.352 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T19:28:01.352 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T19:28:01.353 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T19:28:01.353 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T19:28:01.354 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T19:28:01.736 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:01.748 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":81872},{"type":"v1","addr":"172.21.15.105:6833","nonce":81872}]},"active_addr":"172.21.15.105:6833/81872","active_change":"2023-01-24T19:27:54.097424+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1952787887}]},{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":593041651}]},{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":102804790}]}]}} 2023-01-24T19:28:01.752 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T19:28:01.752 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T19:28:01.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:28:02.069 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:02.070 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"f98da7d2-71a8-4fa9-8756-fa9940165f36","created":"2023-01-24T19:27:48.609675+0000","modified":"2023-01-24T19:28:01.327729+0000","last_up_change":"2023-01-24T19:27:58.113682+0000","last_in_change":"2023-01-24T19:27:51.112886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T19:27:59.910764+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T19:28:00.155040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2e461ba5-43f1-46b1-b762-74211e41888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":82458},{"type":"v1","addr":"172.21.15.62:6801","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":82458},{"type":"v1","addr":"172.21.15.62:6803","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":82458},{"type":"v1","addr":"172.21.15.62:6807","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":82458},{"type":"v1","addr":"172.21.15.62:6805","nonce":82458}]},"public_addr":"172.21.15.62:6801/82458","cluster_addr":"172.21.15.62:6803/82458","heartbeat_back_addr":"172.21.15.62:6807/82458","heartbeat_front_addr":"172.21.15.62:6805/82458","state":["exists","up"]},{"osd":1,"uuid":"ccfe4647-f69d-443a-a543-43082d69dae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":82464},{"type":"v1","addr":"172.21.15.62:6809","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":82464},{"type":"v1","addr":"172.21.15.62:6811","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":82464},{"type":"v1","addr":"172.21.15.62:6815","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":82464},{"type":"v1","addr":"172.21.15.62:6813","nonce":82464}]},"public_addr":"172.21.15.62:6809/82464","cluster_addr":"172.21.15.62:6811/82464","heartbeat_back_addr":"172.21.15.62:6815/82464","heartbeat_front_addr":"172.21.15.62:6813/82464","state":["exists","up"]},{"osd":2,"uuid":"5ece0928-7625-452c-9bb2-b4e7d8c66bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":82483},{"type":"v1","addr":"172.21.15.62:6817","nonce":82483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":82483},{"type":"v1","addr":"172.21.15.62:6819","nonce":82483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":82483},{"type":"v1","addr":"172.21.15.62:6823","nonce":82483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":82483},{"type":"v1","addr":"172.21.15.62:6821","nonce":82483}]},"public_addr":"172.21.15.62:6817/82483","cluster_addr":"172.21.15.62:6819/82483","heartbeat_back_addr":"172.21.15.62:6823/82483","heartbeat_front_addr":"172.21.15.62:6821/82483","state":["exists","up"]},{"osd":3,"uuid":"977e3c9e-7585-45de-a899-41d3e8690a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":82492},{"type":"v1","addr":"172.21.15.62:6825","nonce":82492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":82492},{"type":"v1","addr":"172.21.15.62:6827","nonce":82492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":82492},{"type":"v1","addr":"172.21.15.62:6831","nonce":82492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":82492},{"type":"v1","addr":"172.21.15.62:6829","nonce":82492}]},"public_addr":"172.21.15.62:6825/82492","cluster_addr":"172.21.15.62:6827/82492","heartbeat_back_addr":"172.21.15.62:6831/82492","heartbeat_front_addr":"172.21.15.62:6829/82492","state":["exists","up"]},{"osd":4,"uuid":"acff091a-d719-45ee-934a-e006744a6d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":82421},{"type":"v1","addr":"172.21.15.105:6817","nonce":82421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":82421},{"type":"v1","addr":"172.21.15.105:6819","nonce":82421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":82421},{"type":"v1","addr":"172.21.15.105:6823","nonce":82421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":82421},{"type":"v1","addr":"172.21.15.105:6821","nonce":82421}]},"public_addr":"172.21.15.105:6817/82421","cluster_addr":"172.21.15.105:6819/82421","heartbeat_back_addr":"172.21.15.105:6823/82421","heartbeat_front_addr":"172.21.15.105:6821/82421","state":["exists","up"]},{"osd":5,"uuid":"ade74c56-2106-4a4f-b9de-0f779c624290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":82419},{"type":"v1","addr":"172.21.15.105:6801","nonce":82419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":82419},{"type":"v1","addr":"172.21.15.105:6803","nonce":82419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":82419},{"type":"v1","addr":"172.21.15.105:6807","nonce":82419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":82419},{"type":"v1","addr":"172.21.15.105:6805","nonce":82419}]},"public_addr":"172.21.15.105:6801/82419","cluster_addr":"172.21.15.105:6803/82419","heartbeat_back_addr":"172.21.15.105:6807/82419","heartbeat_front_addr":"172.21.15.105:6805/82419","state":["exists","up"]},{"osd":6,"uuid":"f0859447-fa75-46cb-ad6e-640fdf2ba9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":82436},{"type":"v1","addr":"172.21.15.105:6825","nonce":82436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":82436},{"type":"v1","addr":"172.21.15.105:6827","nonce":82436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":82436},{"type":"v1","addr":"172.21.15.105:6831","nonce":82436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":82436},{"type":"v1","addr":"172.21.15.105:6829","nonce":82436}]},"public_addr":"172.21.15.105:6825/82436","cluster_addr":"172.21.15.105:6827/82436","heartbeat_back_addr":"172.21.15.105:6831/82436","heartbeat_front_addr":"172.21.15.105:6829/82436","state":["exists","up"]},{"osd":7,"uuid":"02f33469-e62e-4f3a-94e1-537535e69e2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":82420},{"type":"v1","addr":"172.21.15.105:6809","nonce":82420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":82420},{"type":"v1","addr":"172.21.15.105:6811","nonce":82420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":82420},{"type":"v1","addr":"172.21.15.105:6815","nonce":82420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":82420},{"type":"v1","addr":"172.21.15.105:6813","nonce":82420}]},"public_addr":"172.21.15.105:6809/82420","cluster_addr":"172.21.15.105:6811/82420","heartbeat_back_addr":"172.21.15.105:6815/82420","heartbeat_front_addr":"172.21.15.105:6813/82420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.264706+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.246881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.319854+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.253673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.298711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.227684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.357601+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.343005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:28:02.083 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T19:28:02.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:28:02.415 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:02.416 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"fsid":"f98da7d2-71a8-4fa9-8756-fa9940165f36","created":"2023-01-24T19:27:48.609675+0000","modified":"2023-01-24T19:28:02.329507+0000","last_up_change":"2023-01-24T19:27:58.113682+0000","last_in_change":"2023-01-24T19:27:51.112886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T19:27:59.910764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T19:28:00.155040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"2e461ba5-43f1-46b1-b762-74211e41888b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":82458},{"type":"v1","addr":"172.21.15.62:6801","nonce":82458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":82458},{"type":"v1","addr":"172.21.15.62:6803","nonce":82458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":82458},{"type":"v1","addr":"172.21.15.62:6807","nonce":82458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":82458},{"type":"v1","addr":"172.21.15.62:6805","nonce":82458}]},"public_addr":"172.21.15.62:6801/82458","cluster_addr":"172.21.15.62:6803/82458","heartbeat_back_addr":"172.21.15.62:6807/82458","heartbeat_front_addr":"172.21.15.62:6805/82458","state":["exists","up"]},{"osd":1,"uuid":"ccfe4647-f69d-443a-a543-43082d69dae1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":82464},{"type":"v1","addr":"172.21.15.62:6809","nonce":82464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":82464},{"type":"v1","addr":"172.21.15.62:6811","nonce":82464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":82464},{"type":"v1","addr":"172.21.15.62:6815","nonce":82464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":82464},{"type":"v1","addr":"172.21.15.62:6813","nonce":82464}]},"public_addr":"172.21.15.62:6809/82464","cluster_addr":"172.21.15.62:6811/82464","heartbeat_back_addr":"172.21.15.62:6815/82464","heartbeat_front_addr":"172.21.15.62:6813/82464","state":["exists","up"]},{"osd":2,"uuid":"5ece0928-7625-452c-9bb2-b4e7d8c66bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":82483},{"type":"v1","addr":"172.21.15.62:6817","nonce":82483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":82483},{"type":"v1","addr":"172.21.15.62:6819","nonce":82483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":82483},{"type":"v1","addr":"172.21.15.62:6823","nonce":82483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":82483},{"type":"v1","addr":"172.21.15.62:6821","nonce":82483}]},"public_addr":"172.21.15.62:6817/82483","cluster_addr":"172.21.15.62:6819/82483","heartbeat_back_addr":"172.21.15.62:6823/82483","heartbeat_front_addr":"172.21.15.62:6821/82483","state":["exists","up"]},{"osd":3,"uuid":"977e3c9e-7585-45de-a899-41d3e8690a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":82492},{"type":"v1","addr":"172.21.15.62:6825","nonce":82492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":82492},{"type":"v1","addr":"172.21.15.62:6827","nonce":82492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":82492},{"type":"v1","addr":"172.21.15.62:6831","nonce":82492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":82492},{"type":"v1","addr":"172.21.15.62:6829","nonce":82492}]},"public_addr":"172.21.15.62:6825/82492","cluster_addr":"172.21.15.62:6827/82492","heartbeat_back_addr":"172.21.15.62:6831/82492","heartbeat_front_addr":"172.21.15.62:6829/82492","state":["exists","up"]},{"osd":4,"uuid":"acff091a-d719-45ee-934a-e006744a6d17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":82421},{"type":"v1","addr":"172.21.15.105:6817","nonce":82421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":82421},{"type":"v1","addr":"172.21.15.105:6819","nonce":82421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":82421},{"type":"v1","addr":"172.21.15.105:6823","nonce":82421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":82421},{"type":"v1","addr":"172.21.15.105:6821","nonce":82421}]},"public_addr":"172.21.15.105:6817/82421","cluster_addr":"172.21.15.105:6819/82421","heartbeat_back_addr":"172.21.15.105:6823/82421","heartbeat_front_addr":"172.21.15.105:6821/82421","state":["exists","up"]},{"osd":5,"uuid":"ade74c56-2106-4a4f-b9de-0f779c624290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":82419},{"type":"v1","addr":"172.21.15.105:6801","nonce":82419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":82419},{"type":"v1","addr":"172.21.15.105:6803","nonce":82419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":82419},{"type":"v1","addr":"172.21.15.105:6807","nonce":82419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":82419},{"type":"v1","addr":"172.21.15.105:6805","nonce":82419}]},"public_addr":"172.21.15.105:6801/82419","cluster_addr":"172.21.15.105:6803/82419","heartbeat_back_addr":"172.21.15.105:6807/82419","heartbeat_front_addr":"172.21.15.105:6805/82419","state":["exists","up"]},{"osd":6,"uuid":"f0859447-fa75-46cb-ad6e-640fdf2ba9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":82436},{"type":"v1","addr":"172.21.15.105:6825","nonce":82436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":82436},{"type":"v1","addr":"172.21.15.105:6827","nonce":82436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":82436},{"type":"v1","addr":"172.21.15.105:6831","nonce":82436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":82436},{"type":"v1","addr":"172.21.15.105:6829","nonce":82436}]},"public_addr":"172.21.15.105:6825/82436","cluster_addr":"172.21.15.105:6827/82436","heartbeat_back_addr":"172.21.15.105:6831/82436","heartbeat_front_addr":"172.21.15.105:6829/82436","state":["exists","up"]},{"osd":7,"uuid":"02f33469-e62e-4f3a-94e1-537535e69e2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":82420},{"type":"v1","addr":"172.21.15.105:6809","nonce":82420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":82420},{"type":"v1","addr":"172.21.15.105:6811","nonce":82420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":82420},{"type":"v1","addr":"172.21.15.105:6815","nonce":82420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":82420},{"type":"v1","addr":"172.21.15.105:6813","nonce":82420}]},"public_addr":"172.21.15.105:6809/82420","cluster_addr":"172.21.15.105:6811/82420","heartbeat_back_addr":"172.21.15.105:6815/82420","heartbeat_front_addr":"172.21.15.105:6813/82420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.264706+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.246881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.319854+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.253673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.298711+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.227684+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.357601+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:27:56.343005+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:28:02.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T19:28:02.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T19:28:02.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T19:28:02.431 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T19:28:02.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T19:28:02.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T19:28:02.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T19:28:02.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T19:28:02.710 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.710 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T19:28:02.720 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T19:28:02.721 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T19:28:02.722 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T19:28:02.724 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2023-01-24T19:28:02.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T19:28:02.727 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:02.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T19:28:02.729 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T19:28:02.735 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:02.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T19:28:03.294 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.299 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:03.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-24T19:28:03.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2023-01-24T19:28:03.337 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.345 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2023-01-24T19:28:03.365 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.366 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-24T19:28:03.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-24T19:28:03.371 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2023-01-24T19:28:03.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2023-01-24T19:28:03.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-24T19:28:03.394 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:03.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2023-01-24T19:28:04.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T19:28:04.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T19:28:04.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T19:28:04.371 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T19:28:04.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T19:28:04.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T19:28:04.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T19:28:04.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T19:28:04.909 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:04.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-24T19:28:04.936 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:04.976 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2023-01-24T19:28:04.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.1 2023-01-24T19:28:04.998 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.008 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2023-01-24T19:28:05.010 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-24T19:28:05.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.4 2023-01-24T19:28:05.033 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.034 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2023-01-24T19:28:05.034 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.042 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2023-01-24T19:28:05.057 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:05.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2023-01-24T19:28:05.061 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2023-01-24T19:28:05.073 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.080 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-24T19:28:05.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2023-01-24T19:28:05.094 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.104 INFO:teuthology.orchestra.run.smithi062.stdout:55834574852 2023-01-24T19:28:05.118 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.0 2023-01-24T19:28:05.119 DEBUG:teuthology.parallel:result is None 2023-01-24T19:28:05.119 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T19:28:05.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:28:05.489 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-24T19:28:05.487+0000 7f6145b09700 0 --2- 172.21.15.62:0/3349572948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f614006c820 0x7f614014e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-01-24T19:28:05.684 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:05.686 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-24T19:28:05.697 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T19:28:04.103368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41120,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":754933600,"statfs":{"total":773094113280,"available":773052006400,"internally_reserved":0,"allocated":1671168,"data_stored":778230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.343814+0000","last_change":"2023-01-24T19:28:01.343814+0000","last_active":"2023-01-24T19:28:01.343814+0000","last_peered":"2023-01-24T19:28:01.343814+0000","last_clean":"2023-01-24T19:28:01.343814+0000","last_became_active":"2023-01-24T19:28:01.343508+0000","last_became_peered":"2023-01-24T19:28:01.343508+0000","last_unstale":"2023-01-24T19:28:01.343814+0000","last_undegraded":"2023-01-24T19:28:01.343814+0000","last_fullsized":"2023-01-24T19:28:01.343814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:07.803886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184957+0000","last_change":"2023-01-24T19:28:02.184957+0000","last_active":"2023-01-24T19:28:02.184957+0000","last_peered":"2023-01-24T19:28:02.184957+0000","last_clean":"2023-01-24T19:28:02.184957+0000","last_became_active":"2023-01-24T19:28:02.184437+0000","last_became_peered":"2023-01-24T19:28:02.184437+0000","last_unstale":"2023-01-24T19:28:02.184957+0000","last_undegraded":"2023-01-24T19:28:02.184957+0000","last_fullsized":"2023-01-24T19:28:02.184957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:22.123733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184918+0000","last_change":"2023-01-24T19:28:02.184918+0000","last_active":"2023-01-24T19:28:02.184918+0000","last_peered":"2023-01-24T19:28:02.184918+0000","last_clean":"2023-01-24T19:28:02.184918+0000","last_became_active":"2023-01-24T19:28:02.184537+0000","last_became_peered":"2023-01-24T19:28:02.184537+0000","last_unstale":"2023-01-24T19:28:02.184918+0000","last_undegraded":"2023-01-24T19:28:02.184918+0000","last_fullsized":"2023-01-24T19:28:02.184918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.139163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.185882+0000","last_change":"2023-01-24T19:28:02.185882+0000","last_active":"2023-01-24T19:28:02.185882+0000","last_peered":"2023-01-24T19:28:02.185882+0000","last_clean":"2023-01-24T19:28:02.185882+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.185882+0000","last_undegraded":"2023-01-24T19:28:02.185882+0000","last_fullsized":"2023-01-24T19:28:02.185882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:11.201201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184856+0000","last_change":"2023-01-24T19:28:02.184856+0000","last_active":"2023-01-24T19:28:02.184856+0000","last_peered":"2023-01-24T19:28:02.184856+0000","last_clean":"2023-01-24T19:28:02.184856+0000","last_became_active":"2023-01-24T19:28:02.184497+0000","last_became_peered":"2023-01-24T19:28:02.184497+0000","last_unstale":"2023-01-24T19:28:02.184856+0000","last_undegraded":"2023-01-24T19:28:02.184856+0000","last_fullsized":"2023-01-24T19:28:02.184856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:12.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'28","reported_seq":47,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.265797+0000","last_change":"2023-01-24T19:28:02.185765+0000","last_active":"2023-01-24T19:28:02.265797+0000","last_peered":"2023-01-24T19:28:02.265797+0000","last_clean":"2023-01-24T19:28:02.265797+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.265797+0000","last_undegraded":"2023-01-24T19:28:02.265797+0000","last_fullsized":"2023-01-24T19:28:02.265797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.737996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.341221+0000","last_change":"2023-01-24T19:28:01.341221+0000","last_active":"2023-01-24T19:28:01.341221+0000","last_peered":"2023-01-24T19:28:01.341221+0000","last_clean":"2023-01-24T19:28:01.341221+0000","last_became_active":"2023-01-24T19:28:01.340904+0000","last_became_peered":"2023-01-24T19:28:01.340904+0000","last_unstale":"2023-01-24T19:28:01.341221+0000","last_undegraded":"2023-01-24T19:28:01.341221+0000","last_fullsized":"2023-01-24T19:28:01.341221+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:08.858575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.345735+0000","last_change":"2023-01-24T19:28:01.344247+0000","last_active":"2023-01-24T19:28:01.345735+0000","last_peered":"2023-01-24T19:28:01.345735+0000","last_clean":"2023-01-24T19:28:01.345735+0000","last_became_active":"2023-01-24T19:28:01.343914+0000","last_became_peered":"2023-01-24T19:28:01.343914+0000","last_unstale":"2023-01-24T19:28:01.345735+0000","last_undegraded":"2023-01-24T19:28:01.345735+0000","last_fullsized":"2023-01-24T19:28:01.345735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:18.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.407368+0000","last_change":"2023-01-24T19:28:01.406840+0000","last_active":"2023-01-24T19:28:01.407368+0000","last_peered":"2023-01-24T19:28:01.407368+0000","last_clean":"2023-01-24T19:28:01.407368+0000","last_became_active":"2023-01-24T19:28:01.406518+0000","last_became_peered":"2023-01-24T19:28:01.406518+0000","last_unstale":"2023-01-24T19:28:01.407368+0000","last_undegraded":"2023-01-24T19:28:01.407368+0000","last_fullsized":"2023-01-24T19:28:01.407368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:24.119085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":549932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5420,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366420,"statfs":{"total":96636764160,"available":96631214080,"internally_reserved":0,"allocated":299008,"data_stored":185486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9260000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0970000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":139276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:28:05.699 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:28:06.015 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:06.017 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-24T19:28:06.028 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T19:28:04.103368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41120,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":754933600,"statfs":{"total":773094113280,"available":773052006400,"internally_reserved":0,"allocated":1671168,"data_stored":778230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.343814+0000","last_change":"2023-01-24T19:28:01.343814+0000","last_active":"2023-01-24T19:28:01.343814+0000","last_peered":"2023-01-24T19:28:01.343814+0000","last_clean":"2023-01-24T19:28:01.343814+0000","last_became_active":"2023-01-24T19:28:01.343508+0000","last_became_peered":"2023-01-24T19:28:01.343508+0000","last_unstale":"2023-01-24T19:28:01.343814+0000","last_undegraded":"2023-01-24T19:28:01.343814+0000","last_fullsized":"2023-01-24T19:28:01.343814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:07.803886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184957+0000","last_change":"2023-01-24T19:28:02.184957+0000","last_active":"2023-01-24T19:28:02.184957+0000","last_peered":"2023-01-24T19:28:02.184957+0000","last_clean":"2023-01-24T19:28:02.184957+0000","last_became_active":"2023-01-24T19:28:02.184437+0000","last_became_peered":"2023-01-24T19:28:02.184437+0000","last_unstale":"2023-01-24T19:28:02.184957+0000","last_undegraded":"2023-01-24T19:28:02.184957+0000","last_fullsized":"2023-01-24T19:28:02.184957+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:22.123733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184918+0000","last_change":"2023-01-24T19:28:02.184918+0000","last_active":"2023-01-24T19:28:02.184918+0000","last_peered":"2023-01-24T19:28:02.184918+0000","last_clean":"2023-01-24T19:28:02.184918+0000","last_became_active":"2023-01-24T19:28:02.184537+0000","last_became_peered":"2023-01-24T19:28:02.184537+0000","last_unstale":"2023-01-24T19:28:02.184918+0000","last_undegraded":"2023-01-24T19:28:02.184918+0000","last_fullsized":"2023-01-24T19:28:02.184918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.139163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.185882+0000","last_change":"2023-01-24T19:28:02.185882+0000","last_active":"2023-01-24T19:28:02.185882+0000","last_peered":"2023-01-24T19:28:02.185882+0000","last_clean":"2023-01-24T19:28:02.185882+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.185882+0000","last_undegraded":"2023-01-24T19:28:02.185882+0000","last_fullsized":"2023-01-24T19:28:02.185882+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:11.201201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.184856+0000","last_change":"2023-01-24T19:28:02.184856+0000","last_active":"2023-01-24T19:28:02.184856+0000","last_peered":"2023-01-24T19:28:02.184856+0000","last_clean":"2023-01-24T19:28:02.184856+0000","last_became_active":"2023-01-24T19:28:02.184497+0000","last_became_peered":"2023-01-24T19:28:02.184497+0000","last_unstale":"2023-01-24T19:28:02.184856+0000","last_undegraded":"2023-01-24T19:28:02.184856+0000","last_fullsized":"2023-01-24T19:28:02.184856+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:12.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'28","reported_seq":47,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.265797+0000","last_change":"2023-01-24T19:28:02.185765+0000","last_active":"2023-01-24T19:28:02.265797+0000","last_peered":"2023-01-24T19:28:02.265797+0000","last_clean":"2023-01-24T19:28:02.265797+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.265797+0000","last_undegraded":"2023-01-24T19:28:02.265797+0000","last_fullsized":"2023-01-24T19:28:02.265797+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.737996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.341221+0000","last_change":"2023-01-24T19:28:01.341221+0000","last_active":"2023-01-24T19:28:01.341221+0000","last_peered":"2023-01-24T19:28:01.341221+0000","last_clean":"2023-01-24T19:28:01.341221+0000","last_became_active":"2023-01-24T19:28:01.340904+0000","last_became_peered":"2023-01-24T19:28:01.340904+0000","last_unstale":"2023-01-24T19:28:01.341221+0000","last_undegraded":"2023-01-24T19:28:01.341221+0000","last_fullsized":"2023-01-24T19:28:01.341221+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:08.858575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.345735+0000","last_change":"2023-01-24T19:28:01.344247+0000","last_active":"2023-01-24T19:28:01.345735+0000","last_peered":"2023-01-24T19:28:01.345735+0000","last_clean":"2023-01-24T19:28:01.345735+0000","last_became_active":"2023-01-24T19:28:01.343914+0000","last_became_peered":"2023-01-24T19:28:01.343914+0000","last_unstale":"2023-01-24T19:28:01.345735+0000","last_undegraded":"2023-01-24T19:28:01.345735+0000","last_fullsized":"2023-01-24T19:28:01.345735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:18.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.407368+0000","last_change":"2023-01-24T19:28:01.406840+0000","last_active":"2023-01-24T19:28:01.407368+0000","last_peered":"2023-01-24T19:28:01.407368+0000","last_clean":"2023-01-24T19:28:01.407368+0000","last_became_active":"2023-01-24T19:28:01.406518+0000","last_became_peered":"2023-01-24T19:28:01.406518+0000","last_unstale":"2023-01-24T19:28:01.407368+0000","last_undegraded":"2023-01-24T19:28:01.407368+0000","last_fullsized":"2023-01-24T19:28:01.407368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:24.119085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":262672,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":52,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":549932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5420,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366420,"statfs":{"total":96636764160,"available":96631214080,"internally_reserved":0,"allocated":299008,"data_stored":185486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9260000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0970000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":139276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:28:06.030 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T19:28:06.030 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T19:28:06.030 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T19:28:06.031 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T19:28:06.428 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:06.428 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T19:28:06.442 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T19:28:06.442 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-24T19:28:06.452 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-24T19:28:06.452 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore odsync write': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 2, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 0.5, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T19:28:06.453 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 2, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-24T19:28:06.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-24T19:28:06.799 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-24T19:28:06.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T19:28:06.800 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:28:07.116 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:07.118 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-24T19:28:07.129 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T19:28:06.103950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41464,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":39488,"kb_avail":754933256,"statfs":{"total":773094113280,"available":773051654144,"internally_reserved":0,"allocated":1761280,"data_stored":856494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40435712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196608,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":47,"num_write_kb":718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000582"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.348700+0000","last_change":"2023-01-24T19:28:01.343814+0000","last_active":"2023-01-24T19:28:01.348700+0000","last_peered":"2023-01-24T19:28:01.348700+0000","last_clean":"2023-01-24T19:28:01.348700+0000","last_became_active":"2023-01-24T19:28:01.343508+0000","last_became_peered":"2023-01-24T19:28:01.343508+0000","last_unstale":"2023-01-24T19:28:01.348700+0000","last_undegraded":"2023-01-24T19:28:01.348700+0000","last_fullsized":"2023-01-24T19:28:01.348700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:07.803886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.188095+0000","last_change":"2023-01-24T19:28:02.184957+0000","last_active":"2023-01-24T19:28:02.188095+0000","last_peered":"2023-01-24T19:28:02.188095+0000","last_clean":"2023-01-24T19:28:02.188095+0000","last_became_active":"2023-01-24T19:28:02.184437+0000","last_became_peered":"2023-01-24T19:28:02.184437+0000","last_unstale":"2023-01-24T19:28:02.188095+0000","last_undegraded":"2023-01-24T19:28:02.188095+0000","last_fullsized":"2023-01-24T19:28:02.188095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:22.123733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.186211+0000","last_change":"2023-01-24T19:28:02.184918+0000","last_active":"2023-01-24T19:28:02.186211+0000","last_peered":"2023-01-24T19:28:02.186211+0000","last_clean":"2023-01-24T19:28:02.186211+0000","last_became_active":"2023-01-24T19:28:02.184537+0000","last_became_peered":"2023-01-24T19:28:02.184537+0000","last_unstale":"2023-01-24T19:28:02.186211+0000","last_undegraded":"2023-01-24T19:28:02.186211+0000","last_fullsized":"2023-01-24T19:28:02.186211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.139163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.188430+0000","last_change":"2023-01-24T19:28:02.185882+0000","last_active":"2023-01-24T19:28:02.188430+0000","last_peered":"2023-01-24T19:28:02.188430+0000","last_clean":"2023-01-24T19:28:02.188430+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.188430+0000","last_undegraded":"2023-01-24T19:28:02.188430+0000","last_fullsized":"2023-01-24T19:28:02.188430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:11.201201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.186394+0000","last_change":"2023-01-24T19:28:02.184856+0000","last_active":"2023-01-24T19:28:02.186394+0000","last_peered":"2023-01-24T19:28:02.186394+0000","last_clean":"2023-01-24T19:28:02.186394+0000","last_became_active":"2023-01-24T19:28:02.184497+0000","last_became_peered":"2023-01-24T19:28:02.184497+0000","last_unstale":"2023-01-24T19:28:02.186394+0000","last_undegraded":"2023-01-24T19:28:02.186394+0000","last_fullsized":"2023-01-24T19:28:02.186394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:12.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'64","reported_seq":98,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T19:28:04.028257+0000","last_change":"2023-01-24T19:28:02.185765+0000","last_active":"2023-01-24T19:28:04.028257+0000","last_peered":"2023-01-24T19:28:04.028257+0000","last_clean":"2023-01-24T19:28:04.028257+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:04.028257+0000","last_undegraded":"2023-01-24T19:28:04.028257+0000","last_fullsized":"2023-01-24T19:28:04.028257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":64,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.737996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.341221+0000","last_change":"2023-01-24T19:28:01.341221+0000","last_active":"2023-01-24T19:28:01.341221+0000","last_peered":"2023-01-24T19:28:01.341221+0000","last_clean":"2023-01-24T19:28:01.341221+0000","last_became_active":"2023-01-24T19:28:01.340904+0000","last_became_peered":"2023-01-24T19:28:01.340904+0000","last_unstale":"2023-01-24T19:28:01.341221+0000","last_undegraded":"2023-01-24T19:28:01.341221+0000","last_fullsized":"2023-01-24T19:28:01.341221+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:08.858575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.345735+0000","last_change":"2023-01-24T19:28:01.344247+0000","last_active":"2023-01-24T19:28:01.345735+0000","last_peered":"2023-01-24T19:28:01.345735+0000","last_clean":"2023-01-24T19:28:01.345735+0000","last_became_active":"2023-01-24T19:28:01.343914+0000","last_became_peered":"2023-01-24T19:28:01.343914+0000","last_unstale":"2023-01-24T19:28:01.345735+0000","last_undegraded":"2023-01-24T19:28:01.345735+0000","last_fullsized":"2023-01-24T19:28:01.345735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:18.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.407368+0000","last_change":"2023-01-24T19:28:01.406840+0000","last_active":"2023-01-24T19:28:01.407368+0000","last_peered":"2023-01-24T19:28:01.407368+0000","last_clean":"2023-01-24T19:28:01.407368+0000","last_became_active":"2023-01-24T19:28:01.406518+0000","last_became_peered":"2023-01-24T19:28:01.406518+0000","last_unstale":"2023-01-24T19:28:01.407368+0000","last_undegraded":"2023-01-24T19:28:01.407368+0000","last_fullsized":"2023-01-24T19:28:01.407368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:24.119085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":808496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":364544,"data_stored":251538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9260000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":635.18799999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:28:07.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:28:07.449 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-24T19:28:07.452 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-24T19:28:07.463 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-24T19:28:06.103950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41464,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":39488,"kb_avail":754933256,"statfs":{"total":773094113280,"available":773051654144,"internally_reserved":0,"allocated":1761280,"data_stored":856494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40435712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196608,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":47,"num_write_kb":718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000582"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.348700+0000","last_change":"2023-01-24T19:28:01.343814+0000","last_active":"2023-01-24T19:28:01.348700+0000","last_peered":"2023-01-24T19:28:01.348700+0000","last_clean":"2023-01-24T19:28:01.348700+0000","last_became_active":"2023-01-24T19:28:01.343508+0000","last_became_peered":"2023-01-24T19:28:01.343508+0000","last_unstale":"2023-01-24T19:28:01.348700+0000","last_undegraded":"2023-01-24T19:28:01.348700+0000","last_fullsized":"2023-01-24T19:28:01.348700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:07.803886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.188095+0000","last_change":"2023-01-24T19:28:02.184957+0000","last_active":"2023-01-24T19:28:02.188095+0000","last_peered":"2023-01-24T19:28:02.188095+0000","last_clean":"2023-01-24T19:28:02.188095+0000","last_became_active":"2023-01-24T19:28:02.184437+0000","last_became_peered":"2023-01-24T19:28:02.184437+0000","last_unstale":"2023-01-24T19:28:02.188095+0000","last_undegraded":"2023-01-24T19:28:02.188095+0000","last_fullsized":"2023-01-24T19:28:02.188095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:22.123733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.186211+0000","last_change":"2023-01-24T19:28:02.184918+0000","last_active":"2023-01-24T19:28:02.186211+0000","last_peered":"2023-01-24T19:28:02.186211+0000","last_clean":"2023-01-24T19:28:02.186211+0000","last_became_active":"2023-01-24T19:28:02.184537+0000","last_became_peered":"2023-01-24T19:28:02.184537+0000","last_unstale":"2023-01-24T19:28:02.186211+0000","last_undegraded":"2023-01-24T19:28:02.186211+0000","last_fullsized":"2023-01-24T19:28:02.186211+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.139163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.188430+0000","last_change":"2023-01-24T19:28:02.185882+0000","last_active":"2023-01-24T19:28:02.188430+0000","last_peered":"2023-01-24T19:28:02.188430+0000","last_clean":"2023-01-24T19:28:02.188430+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:02.188430+0000","last_undegraded":"2023-01-24T19:28:02.188430+0000","last_fullsized":"2023-01-24T19:28:02.188430+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:11.201201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:02.186394+0000","last_change":"2023-01-24T19:28:02.184856+0000","last_active":"2023-01-24T19:28:02.186394+0000","last_peered":"2023-01-24T19:28:02.186394+0000","last_clean":"2023-01-24T19:28:02.186394+0000","last_became_active":"2023-01-24T19:28:02.184497+0000","last_became_peered":"2023-01-24T19:28:02.184497+0000","last_unstale":"2023-01-24T19:28:02.186394+0000","last_undegraded":"2023-01-24T19:28:02.186394+0000","last_fullsized":"2023-01-24T19:28:02.186394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:12.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'64","reported_seq":98,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T19:28:04.028257+0000","last_change":"2023-01-24T19:28:02.185765+0000","last_active":"2023-01-24T19:28:04.028257+0000","last_peered":"2023-01-24T19:28:04.028257+0000","last_clean":"2023-01-24T19:28:04.028257+0000","last_became_active":"2023-01-24T19:28:02.184600+0000","last_became_peered":"2023-01-24T19:28:02.184600+0000","last_unstale":"2023-01-24T19:28:04.028257+0000","last_undegraded":"2023-01-24T19:28:04.028257+0000","last_fullsized":"2023-01-24T19:28:04.028257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":64,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:23.737996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.341221+0000","last_change":"2023-01-24T19:28:01.341221+0000","last_active":"2023-01-24T19:28:01.341221+0000","last_peered":"2023-01-24T19:28:01.341221+0000","last_clean":"2023-01-24T19:28:01.341221+0000","last_became_active":"2023-01-24T19:28:01.340904+0000","last_became_peered":"2023-01-24T19:28:01.340904+0000","last_unstale":"2023-01-24T19:28:01.341221+0000","last_undegraded":"2023-01-24T19:28:01.341221+0000","last_fullsized":"2023-01-24T19:28:01.341221+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:08.858575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.345735+0000","last_change":"2023-01-24T19:28:01.344247+0000","last_active":"2023-01-24T19:28:01.345735+0000","last_peered":"2023-01-24T19:28:01.345735+0000","last_clean":"2023-01-24T19:28:01.345735+0000","last_became_active":"2023-01-24T19:28:01.343914+0000","last_became_peered":"2023-01-24T19:28:01.343914+0000","last_unstale":"2023-01-24T19:28:01.345735+0000","last_undegraded":"2023-01-24T19:28:01.345735+0000","last_fullsized":"2023-01-24T19:28:01.345735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:18.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:28:01.407368+0000","last_change":"2023-01-24T19:28:01.406840+0000","last_active":"2023-01-24T19:28:01.407368+0000","last_peered":"2023-01-24T19:28:01.407368+0000","last_clean":"2023-01-24T19:28:01.407368+0000","last_became_active":"2023-01-24T19:28:01.406518+0000","last_became_peered":"2023-01-24T19:28:01.406518+0000","last_unstale":"2023-01-24T19:28:01.407368+0000","last_undegraded":"2023-01-24T19:28:01.407368+0000","last_fullsized":"2023-01-24T19:28:01.407368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:28:00.323163+0000","last_clean_scrub_stamp":"2023-01-24T19:28:00.323163+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-24T19:29:24.119085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":808496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":364544,"data_stored":251538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9260000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5088,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366752,"statfs":{"total":96636764160,"available":96631554048,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220,"kb_used_data":284,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366620,"statfs":{"total":96636764160,"available":96631418880,"internally_reserved":0,"allocated":290816,"data_stored":177794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5024,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366816,"statfs":{"total":96636764160,"available":96631619584,"internally_reserved":0,"allocated":155648,"data_stored":46210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5100,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366740,"statfs":{"total":96636764160,"available":96631541760,"internally_reserved":0,"allocated":167936,"data_stored":52316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":635.18799999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":205328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:28:07.464 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T19:28:07.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-24T19:28:07.788 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 17 2023-01-24T19:28:07.788 INFO:teuthology.orchestra.run.smithi062.stdout:fsid f98da7d2-71a8-4fa9-8756-fa9940165f36 2023-01-24T19:28:07.788 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-01-24T19:27:48.609675+0000 2023-01-24T19:28:07.789 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2023-01-24T19:28:02.329507+0000 2023-01-24T19:28:07.789 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-24T19:28:07.789 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 4 2023-01-24T19:28:07.789 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2023-01-24T19:28:07.789 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2023-01-24T19:28:07.790 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2023-01-24T19:28:07.790 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2023-01-24T19:28:07.790 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client jewel 2023-01-24T19:28:07.790 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release quincy 2023-01-24T19:28:07.790 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2023-01-24T19:28:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-24T19:28:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-24T19:28:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 8 2023-01-24T19:28:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/82458,v1:172.21.15.62:6801/82458] [v2:172.21.15.62:6802/82458,v1:172.21.15.62:6803/82458] exists,up 2e461ba5-43f1-46b1-b762-74211e41888b 2023-01-24T19:28:07.791 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/82464,v1:172.21.15.62:6809/82464] [v2:172.21.15.62:6810/82464,v1:172.21.15.62:6811/82464] exists,up ccfe4647-f69d-443a-a543-43082d69dae1 2023-01-24T19:28:07.792 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/82483,v1:172.21.15.62:6817/82483] [v2:172.21.15.62:6818/82483,v1:172.21.15.62:6819/82483] exists,up 5ece0928-7625-452c-9bb2-b4e7d8c66bca 2023-01-24T19:28:07.792 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/82492,v1:172.21.15.62:6825/82492] [v2:172.21.15.62:6826/82492,v1:172.21.15.62:6827/82492] exists,up 977e3c9e-7585-45de-a899-41d3e8690a39 2023-01-24T19:28:07.792 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/82421,v1:172.21.15.105:6817/82421] [v2:172.21.15.105:6818/82421,v1:172.21.15.105:6819/82421] exists,up acff091a-d719-45ee-934a-e006744a6d17 2023-01-24T19:28:07.792 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/82419,v1:172.21.15.105:6801/82419] [v2:172.21.15.105:6802/82419,v1:172.21.15.105:6803/82419] exists,up ade74c56-2106-4a4f-b9de-0f779c624290 2023-01-24T19:28:07.793 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/82436,v1:172.21.15.105:6825/82436] [v2:172.21.15.105:6826/82436,v1:172.21.15.105:6827/82436] exists,up f0859447-fa75-46cb-ad6e-640fdf2ba9d3 2023-01-24T19:28:07.793 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/82420,v1:172.21.15.105:6809/82420] [v2:172.21.15.105:6810/82420,v1:172.21.15.105:6811/82420] exists,up 02f33469-e62e-4f3a-94e1-537535e69e2a 2023-01-24T19:28:07.802 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/82458,v1:172.21.15.62:6801/82458] [v2:172.21.15.62:6802/82458,v1:172.21.15.62:6803/82458] exists,up 2e461ba5-43f1-46b1-b762-74211e41888b', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/82464,v1:172.21.15.62:6809/82464] [v2:172.21.15.62:6810/82464,v1:172.21.15.62:6811/82464] exists,up ccfe4647-f69d-443a-a543-43082d69dae1', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/82483,v1:172.21.15.62:6817/82483] [v2:172.21.15.62:6818/82483,v1:172.21.15.62:6819/82483] exists,up 5ece0928-7625-452c-9bb2-b4e7d8c66bca', 'osd.3 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/82492,v1:172.21.15.62:6825/82492] [v2:172.21.15.62:6826/82492,v1:172.21.15.62:6827/82492] exists,up 977e3c9e-7585-45de-a899-41d3e8690a39', 'osd.4 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/82421,v1:172.21.15.105:6817/82421] [v2:172.21.15.105:6818/82421,v1:172.21.15.105:6819/82421] exists,up acff091a-d719-45ee-934a-e006744a6d17', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/82419,v1:172.21.15.105:6801/82419] [v2:172.21.15.105:6802/82419,v1:172.21.15.105:6803/82419] exists,up ade74c56-2106-4a4f-b9de-0f779c624290', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/82436,v1:172.21.15.105:6825/82436] [v2:172.21.15.105:6826/82436,v1:172.21.15.105:6827/82436] exists,up f0859447-fa75-46cb-ad6e-640fdf2ba9d3', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/82420,v1:172.21.15.105:6809/82420] [v2:172.21.15.105:6810/82420,v1:172.21.15.105:6811/82420] exists,up 02f33469-e62e-4f3a-94e1-537535e69e2a'] 2023-01-24T19:28:07.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-24T19:28:07.921 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-01-24T19:28:07.921 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mon.a", 2023-01-24T19:28:07.921 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster": "ceph", 2023-01-24T19:28:07.922 INFO:teuthology.orchestra.run.smithi062.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-24T19:28:07.922 INFO:teuthology.orchestra.run.smithi062.stdout: "admin_socket_mode": "", 2023-01-24T19:28:07.922 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-24T19:28:07.922 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_client_required": "cephx, none", 2023-01-24T19:28:07.922 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_cluster_required": "cephx", 2023-01-24T19:28:07.923 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_debug": "true", 2023-01-24T19:28:07.923 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-24T19:28:07.923 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-24T19:28:07.923 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_service_required": "cephx", 2023-01-24T19:28:07.923 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-24T19:28:07.924 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_supported": "cephx", 2023-01-24T19:28:07.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio": "true", 2023-01-24T19:28:07.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-24T19:28:07.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_poll_ms": "250", 2023-01-24T19:28:07.924 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_reap_max": "16", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_async_discard": "false", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_block_size": "4096", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio": "false", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-24T19:28:07.925 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_inflight_ios": "false", 2023-01-24T19:28:07.926 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_enable_discard": "false", 2023-01-24T19:28:07.926 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_flock_retry": "3", 2023-01-24T19:28:07.926 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-24T19:28:07.926 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_inject_crash": "0", 2023-01-24T19:28:07.926 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-24T19:28:07.927 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring": "false", 2023-01-24T19:28:07.927 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring_hipri": "false", 2023-01-24T19:28:07.927 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-24T19:28:07.927 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-24T19:28:07.927 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-24T19:28:07.928 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-24T19:28:07.928 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_type": "", 2023-01-24T19:28:07.928 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_alloc_size": "1048576", 2023-01-24T19:28:07.928 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_allocator": "hybrid", 2023-01-24T19:28:07.928 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_buffered_io": "true", 2023-01-24T19:28:07.929 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_for_zeros": "false", 2023-01-24T19:28:07.929 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-24T19:28:07.929 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-24T19:28:07.930 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_compact_log_sync": "false", 2023-01-24T19:28:07.930 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-24T19:28:07.930 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-24T19:28:07.930 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-24T19:28:07.930 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_max_log_runway": "4194304", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_max_prefetch": "1048576", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_min_flush_size": "524288", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_min_log_runway": "1048576", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_replay_recovery": "false", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-24T19:28:07.931 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-24T19:28:07.932 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_sync_write": "false", 2023-01-24T19:28:07.932 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-24T19:28:07.932 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-24T19:28:07.932 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_allocation_from_file": "true", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_allocator": "hybrid", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-24T19:28:07.933 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-24T19:28:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-24T19:28:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-24T19:28:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-24T19:28:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_create": "true", 2023-01-24T19:28:07.934 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_create": "false", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_path": "", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_size": "0", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_path": "", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_preallocate_file": "false", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_size": "107374182400", 2023-01-24T19:28:07.935 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_create": "false", 2023-01-24T19:28:07.936 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_path": "", 2023-01-24T19:28:07.936 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_size": "100663296", 2023-01-24T19:28:07.936 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs": "true", 2023-01-24T19:28:07.936 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-24T19:28:07.936 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-24T19:28:07.937 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-24T19:28:07.937 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-24T19:28:07.937 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T19:28:07.937 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T19:28:07.938 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-24T19:28:07.938 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-24T19:28:07.938 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_autotune": "true", 2023-01-24T19:28:07.938 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-24T19:28:07.938 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-24T19:28:07.939 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-24T19:28:07.939 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-24T19:28:07.939 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size": "0", 2023-01-24T19:28:07.939 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-24T19:28:07.940 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-24T19:28:07.940 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-24T19:28:07.940 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-24T19:28:07.940 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_type": "2q", 2023-01-24T19:28:07.940 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_clone_cow": "true", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-24T19:28:07.941 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-24T19:28:07.942 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-24T19:28:07.942 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-24T19:28:07.942 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_mode": "none", 2023-01-24T19:28:07.942 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-24T19:28:07.942 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_csum_type": "crc32c", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_freelist": "false", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-24T19:28:07.943 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-24T19:28:07.944 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-24T19:28:07.944 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-24T19:28:07.944 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-24T19:28:07.944 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-24T19:28:07.944 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_small_allocations": "0", 2023-01-24T19:28:07.945 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_default_buffered_read": "true", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_default_buffered_write": "false", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-24T19:28:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-24T19:28:07.947 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-24T19:28:07.947 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-24T19:28:07.947 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-24T19:28:07.947 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-24T19:28:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fail_eio": "false", 2023-01-24T19:28:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-24T19:28:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-24T19:28:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-24T19:28:07.948 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-24T19:28:07.949 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-24T19:28:07.949 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-24T19:28:07.949 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mount": "false", 2023-01-24T19:28:07.949 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-24T19:28:07.949 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_umount": "false", 2023-01-24T19:28:07.950 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-24T19:28:07.950 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-24T19:28:07.950 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-24T19:28:07.950 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-24T19:28:07.950 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-24T19:28:07.951 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-24T19:28:07.951 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-24T19:28:07.951 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-24T19:28:07.951 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_ignore_data_csum": "false", 2023-01-24T19:28:07.951 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-24T19:28:07.952 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-24T19:28:07.952 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-24T19:28:07.952 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-24T19:28:07.952 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_op_age": "5.000000", 2023-01-24T19:28:07.952 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_alloc_size": "0", 2023-01-24T19:28:07.953 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size": "0", 2023-01-24T19:28:07.953 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-24T19:28:07.953 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-24T19:28:07.953 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-24T19:28:07.953 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_deferred_txc": "32", 2023-01-24T19:28:07.954 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size": "0", 2023-01-24T19:28:07.954 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-24T19:28:07.954 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-24T19:28:07.954 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_nid_prealloc": "1024", 2023-01-24T19:28:07.954 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_retry_disk_reads": "3", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_cf": "true", 2023-01-24T19:28:07.955 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-24T19:28:07.956 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2023-01-24T19:28:07.956 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-24T19:28:07.956 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-24T19:28:07.956 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-24T19:28:07.957 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-24T19:28:07.957 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_mem": "512", 2023-01-24T19:28:07.957 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-24T19:28:07.957 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-24T19:28:07.957 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-24T19:28:07.958 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-24T19:28:07.958 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-24T19:28:07.958 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-24T19:28:07.958 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-24T19:28:07.958 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_tracing": "false", 2023-01-24T19:28:07.959 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-24T19:28:07.959 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-24T19:28:07.959 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-24T19:28:07.959 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-24T19:28:07.959 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-24T19:28:07.960 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-24T19:28:07.960 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-24T19:28:07.960 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-24T19:28:07.960 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-24T19:28:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_zero_block_detection": "false", 2023-01-24T19:28:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-24T19:28:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-24T19:28:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-24T19:28:07.961 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-24T19:28:07.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-24T19:28:07.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-24T19:28:07.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-24T19:28:07.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-24T19:28:07.962 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-24T19:28:07.963 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-24T19:28:07.963 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-24T19:28:07.963 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-24T19:28:07.963 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-24T19:28:07.963 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_cluster_require_signatures": "false", 2023-01-24T19:28:07.964 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_cluster_require_version": "2", 2023-01-24T19:28:07.964 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_require_signatures": "false", 2023-01-24T19:28:07.964 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_require_version": "2", 2023-01-24T19:28:07.964 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_service_require_signatures": "false", 2023-01-24T19:28:07.965 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_service_require_version": "2", 2023-01-24T19:28:07.965 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_sign_messages": "true", 2023-01-24T19:28:07.965 INFO:teuthology.orchestra.run.smithi062.stdout: "chdir": "", 2023-01-24T19:28:07.965 INFO:teuthology.orchestra.run.smithi062.stdout: "client_acl_type": "", 2023-01-24T19:28:07.965 INFO:teuthology.orchestra.run.smithi062.stdout: "client_asio_thread_count": "2", 2023-01-24T19:28:07.966 INFO:teuthology.orchestra.run.smithi062.stdout: "client_cache_mid": "0.750000", 2023-01-24T19:28:07.966 INFO:teuthology.orchestra.run.smithi062.stdout: "client_cache_size": "16384", 2023-01-24T19:28:07.966 INFO:teuthology.orchestra.run.smithi062.stdout: "client_caps_release_delay": "5", 2023-01-24T19:28:07.966 INFO:teuthology.orchestra.run.smithi062.stdout: "client_check_pool_perm": "true", 2023-01-24T19:28:07.966 INFO:teuthology.orchestra.run.smithi062.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-24T19:28:07.967 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_force_sync_read": "false", 2023-01-24T19:28:07.967 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_getattr_caps": "false", 2023-01-24T19:28:07.967 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_inject_tick_delay": "0", 2023-01-24T19:28:07.967 INFO:teuthology.orchestra.run.smithi062.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-24T19:28:07.967 INFO:teuthology.orchestra.run.smithi062.stdout: "client_die_on_failed_remount": "false", 2023-01-24T19:28:07.968 INFO:teuthology.orchestra.run.smithi062.stdout: "client_dirsize_rbytes": "true", 2023-01-24T19:28:07.968 INFO:teuthology.orchestra.run.smithi062.stdout: "client_force_lazyio": "false", 2023-01-24T19:28:07.968 INFO:teuthology.orchestra.run.smithi062.stdout: "client_fs": "", 2023-01-24T19:28:07.968 INFO:teuthology.orchestra.run.smithi062.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-24T19:28:07.968 INFO:teuthology.orchestra.run.smithi062.stdout: "client_inject_release_failure": "false", 2023-01-24T19:28:07.969 INFO:teuthology.orchestra.run.smithi062.stdout: "client_max_inline_size": "4096", 2023-01-24T19:28:07.969 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mds_namespace": "", 2023-01-24T19:28:07.969 INFO:teuthology.orchestra.run.smithi062.stdout: "client_metadata": "", 2023-01-24T19:28:07.969 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_gid": "-1", 2023-01-24T19:28:07.969 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_timeout": "300", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_uid": "-1", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mountpoint": "/", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_notify_timeout": "10", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc": "true", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_dirty": "104857600", 2023-01-24T19:28:07.970 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-24T19:28:07.971 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_objects": "1000", 2023-01-24T19:28:07.971 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_size": "209715200", 2023-01-24T19:28:07.971 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_target_dirty": "8388608", 2023-01-24T19:28:07.971 INFO:teuthology.orchestra.run.smithi062.stdout: "client_permissions": "true", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_quota": "true", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_quota_df": "true", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_max_bytes": "0", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_max_periods": "4", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_min": "131072", 2023-01-24T19:28:07.972 INFO:teuthology.orchestra.run.smithi062.stdout: "client_reconnect_stale": "false", 2023-01-24T19:28:07.973 INFO:teuthology.orchestra.run.smithi062.stdout: "client_shutdown_timeout": "30", 2023-01-24T19:28:07.973 INFO:teuthology.orchestra.run.smithi062.stdout: "client_snapdir": ".snap", 2023-01-24T19:28:07.973 INFO:teuthology.orchestra.run.smithi062.stdout: "client_tick_interval": "1", 2023-01-24T19:28:07.973 INFO:teuthology.orchestra.run.smithi062.stdout: "client_trace": "", 2023-01-24T19:28:07.973 INFO:teuthology.orchestra.run.smithi062.stdout: "client_try_dentry_invalidate": "false", 2023-01-24T19:28:07.974 INFO:teuthology.orchestra.run.smithi062.stdout: "client_use_faked_inos": "false", 2023-01-24T19:28:07.974 INFO:teuthology.orchestra.run.smithi062.stdout: "client_use_random_mds": "false", 2023-01-24T19:28:07.974 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog": "false", 2023-01-24T19:28:07.974 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog_port": "12201", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_monitors": "default=true", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog": "false", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog_level": "info", 2023-01-24T19:28:07.975 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_addr": "-", 2023-01-24T19:28:07.976 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_network": "", 2023-01-24T19:28:07.976 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_network_interface": "", 2023-01-24T19:28:07.976 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_isal": "false", 2023-01-24T19:28:07.976 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_level": "5", 2023-01-24T19:28:07.976 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_winsize": "-15", 2023-01-24T19:28:07.977 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zstd_level": "1", 2023-01-24T19:28:07.977 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-24T19:28:07.977 INFO:teuthology.orchestra.run.smithi062.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-24T19:28:07.977 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_alien_op_num_threads": "6", 2023-01-24T19:28:07.977 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-24T19:28:07.978 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-24T19:28:07.978 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-24T19:28:07.978 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location": "", 2023-01-24T19:28:07.978 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location_hook": "", 2023-01-24T19:28:07.978 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location_hook_timeout": "10", 2023-01-24T19:28:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: "daemonize": "false", 2023-01-24T19:28:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_alienstore": "0/5", 2023-01-24T19:28:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_allow_any_pool_priority": "false", 2023-01-24T19:28:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asok": "1/5", 2023-01-24T19:28:07.979 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asok_assert_abort": "false", 2023-01-24T19:28:07.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asserts_on_shutdown": "true", 2023-01-24T19:28:07.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_auth": "20/20", 2023-01-24T19:28:07.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bdev": "1/3", 2023-01-24T19:28:07.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bluefs": "1/5", 2023-01-24T19:28:07.980 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bluestore": "1/5", 2023-01-24T19:28:07.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_buffer": "0/1", 2023-01-24T19:28:07.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_ceph_exporter": "1/5", 2023-01-24T19:28:07.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_cephfs_mirror": "0/5", 2023-01-24T19:28:07.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_cephsqlite": "0/5", 2023-01-24T19:28:07.981 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_civetweb": "1/10", 2023-01-24T19:28:07.982 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_client": "0/5", 2023-01-24T19:28:07.982 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_compressor": "1/5", 2023-01-24T19:28:07.982 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_context": "0/1", 2023-01-24T19:28:07.982 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_crush": "1/1", 2023-01-24T19:28:07.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_crypto": "1/5", 2023-01-24T19:28:07.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_deliberately_leak_memory": "false", 2023-01-24T19:28:07.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_disable_randomized_ping": "false", 2023-01-24T19:28:07.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_dpdk": "1/5", 2023-01-24T19:28:07.983 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_eventtrace": "1/5", 2023-01-24T19:28:07.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_filer": "0/1", 2023-01-24T19:28:07.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_filestore": "1/3", 2023-01-24T19:28:07.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_finisher": "1/1", 2023-01-24T19:28:07.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_fuse": "1/5", 2023-01-24T19:28:07.984 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_heartbeat_testing_span": "0", 2023-01-24T19:28:07.985 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_heartbeatmap": "1/5", 2023-01-24T19:28:07.985 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-24T19:28:07.985 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_javaclient": "1/5", 2023-01-24T19:28:07.985 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_journal": "1/3", 2023-01-24T19:28:07.985 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_journaler": "0/5", 2023-01-24T19:28:07.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_kstore": "1/5", 2023-01-24T19:28:07.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_leveldb": "4/5", 2023-01-24T19:28:07.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_lockdep": "0/1", 2023-01-24T19:28:07.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mclock": "1/5", 2023-01-24T19:28:07.986 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds": "1/5", 2023-01-24T19:28:07.987 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_balancer": "1/5", 2023-01-24T19:28:07.987 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_locker": "1/5", 2023-01-24T19:28:07.987 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_log": "1/5", 2023-01-24T19:28:07.987 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_log_expire": "1/5", 2023-01-24T19:28:07.987 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_migrator": "1/5", 2023-01-24T19:28:07.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_memdb": "4/5", 2023-01-24T19:28:07.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mgr": "2/5", 2023-01-24T19:28:07.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mgrc": "1/5", 2023-01-24T19:28:07.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mon": "20/20", 2023-01-24T19:28:07.988 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_monc": "0/10", 2023-01-24T19:28:07.989 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_ms": "1/1", 2023-01-24T19:28:07.989 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_none": "0/5", 2023-01-24T19:28:07.989 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objclass": "0/5", 2023-01-24T19:28:07.989 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objectcacher": "0/5", 2023-01-24T19:28:07.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objecter": "0/1", 2023-01-24T19:28:07.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_optracker": "0/5", 2023-01-24T19:28:07.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_osd": "1/5", 2023-01-24T19:28:07.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_paxos": "20/20", 2023-01-24T19:28:07.990 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_perfcounter": "1/5", 2023-01-24T19:28:07.991 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_prioritycache": "1/5", 2023-01-24T19:28:07.991 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rados": "0/5", 2023-01-24T19:28:07.991 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd": "0/5", 2023-01-24T19:28:07.991 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_mirror": "0/5", 2023-01-24T19:28:07.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_pwl": "0/5", 2023-01-24T19:28:07.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_replay": "0/5", 2023-01-24T19:28:07.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_refs": "0/0", 2023-01-24T19:28:07.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_reserver": "1/1", 2023-01-24T19:28:07.992 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw": "1/5", 2023-01-24T19:28:07.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_datacache": "1/5", 2023-01-24T19:28:07.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_sync": "1/5", 2023-01-24T19:28:07.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rocksdb": "4/5", 2023-01-24T19:28:07.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore": "0/5", 2023-01-24T19:28:07.993 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_cache": "0/5", 2023-01-24T19:28:07.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_cleaner": "0/5", 2023-01-24T19:28:07.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_device": "0/5", 2023-01-24T19:28:07.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_journal": "0/5", 2023-01-24T19:28:07.994 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_lba": "0/5", 2023-01-24T19:28:07.995 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_odata": "0/5", 2023-01-24T19:28:07.995 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_omap": "0/5", 2023-01-24T19:28:07.995 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_onode": "0/5", 2023-01-24T19:28:07.995 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_tm": "0/5", 2023-01-24T19:28:07.995 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_striper": "0/1", 2023-01-24T19:28:07.996 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_test": "0/5", 2023-01-24T19:28:07.996 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_throttle": "1/1", 2023-01-24T19:28:07.996 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_timer": "0/1", 2023-01-24T19:28:07.996 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_tp": "0/5", 2023-01-24T19:28:07.996 INFO:teuthology.orchestra.run.smithi062.stdout: "device_failure_prediction_mode": "none", 2023-01-24T19:28:07.997 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-24T19:28:07.997 INFO:teuthology.orchestra.run.smithi062.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-24T19:28:07.997 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_graylog": "false", 2023-01-24T19:28:07.997 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_journald": "false", 2023-01-24T19:28:07.997 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_stderr": "true", 2023-01-24T19:28:07.998 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_syslog": "false", 2023-01-24T19:28:07.998 INFO:teuthology.orchestra.run.smithi062.stdout: "event_tracing": "false", 2023-01-24T19:28:07.998 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_addr": "0.0.0.0", 2023-01-24T19:28:07.998 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_http_port": "9926", 2023-01-24T19:28:07.998 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_prio_limit": "5", 2023-01-24T19:28:07.999 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-24T19:28:07.999 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_sort_metrics": "true", 2023-01-24T19:28:07.999 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_stats_period": "5", 2023-01-24T19:28:07.999 INFO:teuthology.orchestra.run.smithi062.stdout: "fake_statfs_for_testing": "0", 2023-01-24T19:28:07.999 INFO:teuthology.orchestra.run.smithi062.stdout: "fatal_signal_handlers": "true", 2023-01-24T19:28:08.000 INFO:teuthology.orchestra.run.smithi062.stdout: "filer_max_purge_ops": "10", 2023-01-24T19:28:08.000 INFO:teuthology.orchestra.run.smithi062.stdout: "filer_max_truncate_ops": "128", 2023-01-24T19:28:08.000 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_apply_finisher_threads": "1", 2023-01-24T19:28:08.000 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_blackhole": "false", 2023-01-24T19:28:08.000 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_btrfs_clone_range": "true", 2023-01-24T19:28:08.001 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_btrfs_snap": "true", 2023-01-24T19:28:08.001 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_caller_concurrency": "10", 2023-01-24T19:28:08.001 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_collect_device_partition_information": "true", 2023-01-24T19:28:08.001 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_commit_timeout": "600.000000", 2023-01-24T19:28:08.001 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_inject_read_err": "false", 2023-01-24T19:28:08.002 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_omap_check": "false", 2023-01-24T19:28:08.002 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-24T19:28:08.002 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_verify_split": "false", 2023-01-24T19:28:08.002 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_dump_file": "", 2023-01-24T19:28:08.003 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-24T19:28:08.003 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-24T19:28:08.003 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fadvise": "true", 2023-01-24T19:28:08.003 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fail_eio": "true", 2023-01-24T19:28:08.003 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fd_cache_shards": "16", 2023-01-24T19:28:08.004 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fd_cache_size": "128", 2023-01-24T19:28:08.004 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fiemap": "false", 2023-01-24T19:28:08.004 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fiemap_threshold": "4096", 2023-01-24T19:28:08.004 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-24T19:28:08.004 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-24T19:28:08.005 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_inject_stall": "0", 2023-01-24T19:28:08.005 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_parallel": "false", 2023-01-24T19:28:08.005 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_trailing": "false", 2023-01-24T19:28:08.005 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_writeahead": "false", 2023-01-24T19:28:08.005 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_kill_at": "0", 2023-01-24T19:28:08.006 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-24T19:28:08.006 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-24T19:28:08.006 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-24T19:28:08.006 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-24T19:28:08.006 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-24T19:28:08.007 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs": "0", 2023-01-24T19:28:08.007 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-24T19:28:08.007 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-24T19:28:08.007 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-24T19:28:08.007 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-24T19:28:08.008 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size": "0", 2023-01-24T19:28:08.008 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-24T19:28:08.008 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-24T19:28:08.008 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-24T19:28:08.008 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_merge_threshold": "-10", 2023-01-24T19:28:08.009 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-24T19:28:08.009 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_odsync_write": "false", 2023-01-24T19:28:08.009 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_backend": "rocksdb", 2023-01-24T19:28:08.009 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_backend_path": "", 2023-01-24T19:28:08.009 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-24T19:28:08.010 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-24T19:28:08.010 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-24T19:28:08.010 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_thread_timeout": "60", 2023-01-24T19:28:08.010 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_threads": "2", 2023-01-24T19:28:08.010 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_punch_hole": "false", 2023-01-24T19:28:08.011 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-24T19:28:08.011 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-24T19:28:08.011 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-24T19:28:08.011 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-24T19:28:08.011 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-24T19:28:08.012 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-24T19:28:08.012 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-24T19:28:08.012 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-24T19:28:08.012 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-24T19:28:08.013 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_ops": "50", 2023-01-24T19:28:08.013 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-24T19:28:08.013 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_seek_data_hole": "false", 2023-01-24T19:28:08.013 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_sloppy_crc": "false", 2023-01-24T19:28:08.014 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-24T19:28:08.014 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_splice": "false", 2023-01-24T19:28:08.014 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_split_multiple": "2", 2023-01-24T19:28:08.014 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_split_rand_factor": "20", 2023-01-24T19:28:08.014 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_update_to": "1000", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-24T19:28:08.015 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-24T19:28:08.016 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_enable": "true", 2023-01-24T19:28:08.016 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-24T19:28:08.016 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-24T19:28:08.016 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-24T19:28:08.017 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-24T19:28:08.017 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-24T19:28:08.017 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-24T19:28:08.017 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_xfs_extsize": "false", 2023-01-24T19:28:08.017 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_zfs_snap": "false", 2023-01-24T19:28:08.018 INFO:teuthology.orchestra.run.smithi062.stdout: "fio_dir": "/tmp/fio", 2023-01-24T19:28:08.018 INFO:teuthology.orchestra.run.smithi062.stdout: "fsid": "f98da7d2-71a8-4fa9-8756-fa9940165f36", 2023-01-24T19:28:08.018 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_allow_other": "true", 2023-01-24T19:28:08.018 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_atomic_o_trunc": "true", 2023-01-24T19:28:08.018 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_big_writes": "true", 2023-01-24T19:28:08.019 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_debug": "false", 2023-01-24T19:28:08.019 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_default_permissions": "false", 2023-01-24T19:28:08.019 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_disable_pagecache": "false", 2023-01-24T19:28:08.019 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_max_write": "0", 2023-01-24T19:28:08.019 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_multithreaded": "true", 2023-01-24T19:28:08.020 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_require_active_mds": "true", 2023-01-24T19:28:08.020 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_set_user_groups": "true", 2023-01-24T19:28:08.020 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_move": "true", 2023-01-24T19:28:08.020 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_read": "true", 2023-01-24T19:28:08.020 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_write": "true", 2023-01-24T19:28:08.021 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-24T19:28:08.021 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_use_invalidate_cb": "true", 2023-01-24T19:28:08.021 INFO:teuthology.orchestra.run.smithi062.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-24T19:28:08.021 INFO:teuthology.orchestra.run.smithi062.stdout: "gss_target_name": "ceph", 2023-01-24T19:28:08.022 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_file": "", 2023-01-24T19:28:08.022 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_inject_failure": "0", 2023-01-24T19:28:08.022 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_interval": "5", 2023-01-24T19:28:08.022 INFO:teuthology.orchestra.run.smithi062.stdout: "host": "smithi062", 2023-01-24T19:28:08.022 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-24T19:28:08.023 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-24T19:28:08.023 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-24T19:28:08.023 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-24T19:28:08.023 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-24T19:28:08.023 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-24T19:28:08.024 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-24T19:28:08.024 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-24T19:28:08.024 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-24T19:28:08.024 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-24T19:28:08.024 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-24T19:28:08.025 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-24T19:28:08.025 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-24T19:28:08.025 INFO:teuthology.orchestra.run.smithi062.stdout: "inject_early_sigterm": "false", 2023-01-24T19:28:08.025 INFO:teuthology.orchestra.run.smithi062.stdout: "jaeger_tracing_enable": "false", 2023-01-24T19:28:08.025 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_aio": "true", 2023-01-24T19:28:08.026 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_align_min_size": "65536", 2023-01-24T19:28:08.026 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_block_align": "true", 2023-01-24T19:28:08.026 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_block_size": "4096", 2023-01-24T19:28:08.026 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_dio": "true", 2023-01-24T19:28:08.026 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_discard": "false", 2023-01-24T19:28:08.027 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_force_aio": "false", 2023-01-24T19:28:08.027 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_ignore_corruption": "false", 2023-01-24T19:28:08.027 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_max_write_bytes": "10485760", 2023-01-24T19:28:08.027 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_max_write_entries": "100", 2023-01-24T19:28:08.027 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_replay_from": "0", 2023-01-24T19:28:08.028 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-24T19:28:08.028 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-24T19:28:08.028 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-24T19:28:08.028 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-24T19:28:08.028 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_write_header_frequency": "0", 2023-01-24T19:28:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_zero_on_create": "false", 2023-01-24T19:28:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_prefetch_periods": "10", 2023-01-24T19:28:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_prezero_periods": "5", 2023-01-24T19:28:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_write_head_interval": "15", 2023-01-24T19:28:08.029 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "", 2023-01-24T19:28:08.030 INFO:teuthology.orchestra.run.smithi062.stdout: "keyfile": "", 2023-01-24T19:28:08.030 INFO:teuthology.orchestra.run.smithi062.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-24T19:28:08.030 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_backend": "rocksdb", 2023-01-24T19:28:08.030 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_default_stripe_size": "65536", 2023-01-24T19:28:08.030 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_fsck_on_mount": "false", 2023-01-24T19:28:08.031 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-24T19:28:08.031 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_max_bytes": "67108864", 2023-01-24T19:28:08.031 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_max_ops": "512", 2023-01-24T19:28:08.031 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_nid_prealloc": "1024", 2023-01-24T19:28:08.031 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_onode_map_size": "1024", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_sync_submit_transaction": "false", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_sync_transaction": "false", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_block_size": "65536", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_bloom_size": "0", 2023-01-24T19:28:08.032 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_cache_size": "536870912", 2023-01-24T19:28:08.033 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_compact_on_mount": "false", 2023-01-24T19:28:08.033 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_compression": "false", 2023-01-24T19:28:08.033 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_log": "", 2023-01-24T19:28:08.033 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-24T19:28:08.033 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_max_open_files": "0", 2023-01-24T19:28:08.034 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_paranoid": "false", 2023-01-24T19:28:08.034 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-24T19:28:08.034 INFO:teuthology.orchestra.run.smithi062.stdout: "librados_thread_count": "2", 2023-01-24T19:28:08.034 INFO:teuthology.orchestra.run.smithi062.stdout: "lockdep": "false", 2023-01-24T19:28:08.034 INFO:teuthology.orchestra.run.smithi062.stdout: "lockdep_force_backtrace": "false", 2023-01-24T19:28:08.035 INFO:teuthology.orchestra.run.smithi062.stdout: "log_coarse_timestamps": "true", 2023-01-24T19:28:08.035 INFO:teuthology.orchestra.run.smithi062.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-24T19:28:08.035 INFO:teuthology.orchestra.run.smithi062.stdout: "log_flush_on_exit": "false", 2023-01-24T19:28:08.035 INFO:teuthology.orchestra.run.smithi062.stdout: "log_graylog_host": "127.0.0.1", 2023-01-24T19:28:08.035 INFO:teuthology.orchestra.run.smithi062.stdout: "log_graylog_port": "12201", 2023-01-24T19:28:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_max_new": "1000", 2023-01-24T19:28:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_max_recent": "10000", 2023-01-24T19:28:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_stderr_prefix": "", 2023-01-24T19:28:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_stop_at_utilization": "0.970000", 2023-01-24T19:28:08.036 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": "true", 2023-01-24T19:28:08.037 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_graylog": "false", 2023-01-24T19:28:08.037 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_journald": "false", 2023-01-24T19:28:08.037 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_stderr": "false", 2023-01-24T19:28:08.037 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_syslog": "false", 2023-01-24T19:28:08.038 INFO:teuthology.orchestra.run.smithi062.stdout: "max_rotating_auth_attempts": "10", 2023-01-24T19:28:08.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_action_on_write_error": "1", 2023-01-24T19:28:08.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_alternate_name_max": "8192", 2023-01-24T19:28:08.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_asio_thread_count": "2", 2023-01-24T19:28:08.038 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_export_pin": "true", 2023-01-24T19:28:08.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_dirs": "true", 2023-01-24T19:28:08.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-24T19:28:08.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_interval": "5", 2023-01-24T19:28:08.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-24T19:28:08.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-24T19:28:08.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_interval": "10", 2023-01-24T19:28:08.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_max": "-1", 2023-01-24T19:28:08.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_max_until": "-1", 2023-01-24T19:28:08.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_merge_size": "50", 2023-01-24T19:28:08.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_midchunk": "0.300000", 2023-01-24T19:28:08.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-24T19:28:08.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_min_start": "0.200000", 2023-01-24T19:28:08.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_minchunk": "0.001000", 2023-01-24T19:28:08.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_mode": "0", 2023-01-24T19:28:08.041 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_need_max": "1.200000", 2023-01-24T19:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_need_min": "0.800000", 2023-01-24T19:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-24T19:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-24T19:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_bits": "3", 2023-01-24T19:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-24T19:28:08.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_size": "10000", 2023-01-24T19:28:08.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-24T19:28:08.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_target_decay": "10.000000", 2023-01-24T19:28:08.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-24T19:28:08.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_grace": "15.000000", 2023-01-24T19:28:08.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_interval": "4.000000", 2023-01-24T19:28:08.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-24T19:28:08.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-24T19:28:08.044 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_mid": "0.700000", 2023-01-24T19:28:08.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_release_free_interval": "10", 2023-01-24T19:28:08.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_reservation": "0.050000", 2023-01-24T19:28:08.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-24T19:28:08.045 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_interval": "1", 2023-01-24T19:28:08.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_threshold": "262144", 2023-01-24T19:28:08.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-24T19:28:08.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-24T19:28:08.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-24T19:28:08.046 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_prealloc_inos": "1000", 2023-01-24T19:28:08.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-24T19:28:08.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_connect_bootstrapping": "false", 2023-01-24T19:28:08.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_damage_table_max_entries": "10000", 2023-01-24T19:28:08.047 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-24T19:28:08.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_auth_pins": "false", 2023-01-24T19:28:08.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_frag": "false", 2023-01-24T19:28:08.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_scatterstat": "false", 2023-01-24T19:28:08.048 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_subtrees": "false", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_decay_halflife": "5.000000", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_default_dir_hash": "2", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_defer_session_stale": "true", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_deny_all_reconnect": "false", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_keys_per_op": "16384", 2023-01-24T19:28:08.049 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_max_commit_size": "10", 2023-01-24T19:28:08.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_max_entries": "0", 2023-01-24T19:28:08.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-24T19:28:08.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-24T19:28:08.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_on_map": "false", 2023-01-24T19:28:08.050 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-24T19:28:08.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-24T19:28:08.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_early_reply": "true", 2023-01-24T19:28:08.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_enable_op_tracker": "true", 2023-01-24T19:28:08.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_enforce_unique_name": "true", 2023-01-24T19:28:08.051 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-24T19:28:08.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-24T19:28:08.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_random": "true", 2023-01-24T19:28:08.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-24T19:28:08.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-24T19:28:08.052 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_health_summarize_threshold": "10", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-24T19:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_health_dummy": "false", 2023-01-24T19:28:08.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_migrator_session_race": "false", 2023-01-24T19:28:08.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-24T19:28:08.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_join_fs": "", 2023-01-24T19:28:08.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_journal_format": "1", 2023-01-24T19:28:08.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_create_at": "0", 2023-01-24T19:28:08.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_export_at": "0", 2023-01-24T19:28:08.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_import_at": "0", 2023-01-24T19:28:08.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_at": "0", 2023-01-24T19:28:08.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_expire_at": "0", 2023-01-24T19:28:08.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_replay_at": "0", 2023-01-24T19:28:08.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_link_at": "0", 2023-01-24T19:28:08.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_mdstable_at": "0", 2023-01-24T19:28:08.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_openc_at": "0", 2023-01-24T19:28:08.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_rename_at": "0", 2023-01-24T19:28:08.056 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_events_per_segment": "1024", 2023-01-24T19:28:08.057 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_max_events": "-1", 2023-01-24T19:28:08.057 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_max_segments": "128", 2023-01-24T19:28:08.057 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_pause": "false", 2023-01-24T19:28:08.057 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_segment_size": "0", 2023-01-24T19:28:08.057 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-24T19:28:08.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_warn_factor": "2.000000", 2023-01-24T19:28:08.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_caps_per_client": "1048576", 2023-01-24T19:28:08.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_completed_flushes": "100000", 2023-01-24T19:28:08.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_completed_requests": "100000", 2023-01-24T19:28:08.058 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_export_size": "20971520", 2023-01-24T19:28:08.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_file_recover": "32", 2023-01-24T19:28:08.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_files": "64", 2023-01-24T19:28:08.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_ops": "8192", 2023-01-24T19:28:08.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_snaps_per_dir": "100", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_metrics_update_interval": "2", 2023-01-24T19:28:08.060 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_min_caps_per_client": "100", 2023-01-24T19:28:08.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_min_caps_working_set": "10000", 2023-01-24T19:28:08.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-24T19:28:08.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_numa_node": "-1", 2023-01-24T19:28:08.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-24T19:28:08.061 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_complaint_time": "30.000000", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_history_duration": "600", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_history_size": "20", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_log_threshold": "5", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_ping_grace": "15", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_ping_interval": "5", 2023-01-24T19:28:08.062 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_caps": "30000", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-24T19:28:08.063 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_warning_threshold": "262144", 2023-01-24T19:28:08.064 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-24T19:28:08.064 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_replay_interval": "1.000000", 2023-01-24T19:28:08.064 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-24T19:28:08.064 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-24T19:28:08.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_root_ino_gid": "0", 2023-01-24T19:28:08.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_root_ino_uid": "0", 2023-01-24T19:28:08.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-24T19:28:08.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-24T19:28:08.065 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-24T19:28:08.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-24T19:28:08.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-24T19:28:08.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-24T19:28:08.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-24T19:28:08.066 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-24T19:28:08.067 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-24T19:28:08.067 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_shutdown_check": "0", 2023-01-24T19:28:08.067 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_skip_ino": "0", 2023-01-24T19:28:08.067 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-24T19:28:08.067 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_max_uid": "4294967294", 2023-01-24T19:28:08.068 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_min_uid": "0", 2023-01-24T19:28:08.068 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_rstat": "false", 2023-01-24T19:28:08.068 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_standby_replay_damaged": "false", 2023-01-24T19:28:08.068 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_symlink_recovery": "true", 2023-01-24T19:28:08.068 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-24T19:28:08.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_thrash_exports": "0", 2023-01-24T19:28:08.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_thrash_fragments": "0", 2023-01-24T19:28:08.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_tick_interval": "5.000000", 2023-01-24T19:28:08.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_valgrind_exit": "false", 2023-01-24T19:28:08.069 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_verify_backtrace": "1", 2023-01-24T19:28:08.070 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_verify_scatter": "false", 2023-01-24T19:28:08.070 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-24T19:28:08.070 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_wipe_sessions": "false", 2023-01-24T19:28:08.070 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool_debug": "false", 2023-01-24T19:28:08.070 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-24T19:28:08.071 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_device_bytes": "1073741824", 2023-01-24T19:28:08.071 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_page_set": "false", 2023-01-24T19:28:08.071 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_page_size": "65536", 2023-01-24T19:28:08.071 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_bytes": "134217728", 2023-01-24T19:28:08.071 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_messages": "512", 2023-01-24T19:28:08.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-24T19:28:08.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-24T19:28:08.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-24T19:28:08.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-24T19:28:08.072 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_disabled_modules": "", 2023-01-24T19:28:08.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-24T19:28:08.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_max_pg_num_change": "128", 2023-01-24T19:28:08.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mds_bytes": "134217728", 2023-01-24T19:28:08.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mds_messages": "128", 2023-01-24T19:28:08.073 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-24T19:28:08.074 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mon_bytes": "134217728", 2023-01-24T19:28:08.074 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mon_messages": "128", 2023-01-24T19:28:08.074 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_osd_bytes": "536870912", 2023-01-24T19:28:08.074 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_osd_messages": "8192", 2023-01-24T19:28:08.075 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_pool": "true", 2023-01-24T19:28:08.075 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-24T19:28:08.075 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_standby_modules": "true", 2023-01-24T19:28:08.075 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_stats_period": "5", 2023-01-24T19:28:08.076 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_stats_threshold": "5", 2023-01-24T19:28:08.076 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_tick_period": "2", 2023-01-24T19:28:08.076 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-24T19:28:08.076 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-24T19:28:08.076 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_allow_pool_delete": "true", 2023-01-24T19:28:08.077 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_allow_pool_size_one": "true", 2023-01-24T19:28:08.077 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_auth_validate_all_caps": "true", 2023-01-24T19:28:08.077 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-24T19:28:08.077 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-24T19:28:08.077 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_bytes": "104857600", 2023-01-24T19:28:08.078 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_directed_command_retry": "5", 2023-01-24T19:28:08.078 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-24T19:28:08.078 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-24T19:28:08.078 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-24T19:28:08.078 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_parallel": "3", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_log_interval": "1.000000", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_ping_interval": "10.000000", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-24T19:28:08.079 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-24T19:28:08.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-24T19:28:08.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-24T19:28:08.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_file_level": "debug", 2023-01-24T19:28:08.080 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_file": "true", 2023-01-24T19:28:08.081 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-24T19:28:08.081 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-24T19:28:08.081 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-24T19:28:08.081 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_journald": "false", 2023-01-24T19:28:08.081 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-24T19:28:08.082 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-24T19:28:08.082 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-24T19:28:08.082 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-24T19:28:08.082 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_bootstrap": "false", 2023-01-24T19:28:08.082 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_start": "false", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_trim": "true", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cpu_threads": "4", 2023-01-24T19:28:08.083 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_crush_min_required_version": "hammer", 2023-01-24T19:28:08.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_daemon_bytes": "419430400", 2023-01-24T19:28:08.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-24T19:28:08.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_avail_crit": "5", 2023-01-24T19:28:08.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_avail_warn": "5", 2023-01-24T19:28:08.084 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_size_warn": "16106127360", 2023-01-24T19:28:08.085 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-24T19:28:08.085 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-24T19:28:08.085 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_json": "false", 2023-01-24T19:28:08.085 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-24T19:28:08.086 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_transactions": "false", 2023-01-24T19:28:08.086 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_extra_checks": "false", 2023-01-24T19:28:08.086 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-24T19:28:08.086 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-24T19:28:08.086 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_pacific": "false", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_quincy": "false", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_down_mkfs_grace": "120", 2023-01-24T19:28:08.087 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_election_default_strategy": "3", 2023-01-24T19:28:08.088 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_election_timeout": "5.000000", 2023-01-24T19:28:08.088 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-24T19:28:08.088 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ping_divisor": "2", 2023-01-24T19:28:08.088 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-24T19:28:08.088 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_enable_op_tracker": "true", 2023-01-24T19:28:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_fake_pool_delete": "false", 2023-01-24T19:28:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_force_quorum_join": "false", 2023-01-24T19:28:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_globalid_prealloc": "10000", 2023-01-24T19:28:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_detail_to_clog": "false", 2023-01-24T19:28:08.089 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_log_update_period": "5", 2023-01-24T19:28:08.090 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_max_detail": "50", 2023-01-24T19:28:08.090 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog": "true", 2023-01-24T19:28:08.090 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog_interval": "600", 2023-01-24T19:28:08.090 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-24T19:28:08.090 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_host": "172.21.15.62,[v2:172.21.15.62:3301,v1:172.21.15.62:6790],172.21.15.105", 2023-01-24T19:28:08.091 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_host_override": "", 2023-01-24T19:28:08.091 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_initial_members": "", 2023-01-24T19:28:08.091 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-24T19:28:08.091 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-24T19:28:08.092 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-24T19:28:08.092 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-24T19:28:08.092 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-24T19:28:08.092 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease": "5.000000", 2023-01-24T19:28:08.092 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-24T19:28:08.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-24T19:28:08.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_full_interval": "50", 2023-01-24T19:28:08.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_max": "10000", 2023-01-24T19:28:08.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_max_summary": "50", 2023-01-24T19:28:08.093 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-24T19:28:08.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_log_epochs": "500", 2023-01-24T19:28:08.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-24T19:28:08.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-24T19:28:08.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_osd": "10000", 2023-01-24T19:28:08.094 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_pg_per_osd": "10000", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_pool_pg_num": "65536", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_force_trim_to": "0", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_skip_sanity": "false", 2023-01-24T19:28:08.095 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_memory_autotune": "true", 2023-01-24T19:28:08.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_memory_target": "2147483648", 2023-01-24T19:28:08.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_beacon_grace": "30", 2023-01-24T19:28:08.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-24T19:28:08.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_digest_period": "5", 2023-01-24T19:28:08.096 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_inactive_grace": "60", 2023-01-24T19:28:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-24T19:28:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-24T19:28:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_min_osdmap_epochs": "50", 2023-01-24T19:28:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_complaint_time": "30", 2023-01-24T19:28:08.097 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_duration": "600", 2023-01-24T19:28:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_size": "20", 2023-01-24T19:28:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_slow_op_size": "20", 2023-01-24T19:28:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-24T19:28:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_log_threshold": "5", 2023-01-24T19:28:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-24T19:28:08.099 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-24T19:28:08.099 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-24T19:28:08.099 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_in": "false", 2023-01-24T19:28:08.099 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-24T19:28:08.099 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-24T19:28:08.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-24T19:28:08.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_cache_size": "500", 2023-01-24T19:28:08.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-24T19:28:08.100 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-24T19:28:08.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-24T19:28:08.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_down_out_interval": "600", 2023-01-24T19:28:08.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-24T19:28:08.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-24T19:28:08.101 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_force_trim_to": "0", 2023-01-24T19:28:08.102 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-24T19:28:08.102 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-24T19:28:08.102 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-24T19:28:08.102 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-24T19:28:08.102 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-24T19:28:08.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-24T19:28:08.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-24T19:28:08.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-24T19:28:08.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_down_reporters": "2", 2023-01-24T19:28:08.103 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-24T19:28:08.104 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-24T19:28:08.104 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-24T19:28:08.104 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-24T19:28:08.104 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-24T19:28:08.104 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-24T19:28:08.105 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_report_timeout": "900", 2023-01-24T19:28:08.105 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-24T19:28:08.105 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-24T19:28:08.105 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-24T19:28:08.105 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-24T19:28:08.106 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_stuck_threshold": "60", 2023-01-24T19:28:08.107 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-24T19:28:08.107 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-24T19:28:08.107 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-24T19:28:08.107 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-24T19:28:08.107 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-24T19:28:08.108 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-24T19:28:08.108 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_probe_timeout": "2.000000", 2023-01-24T19:28:08.108 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_max_change": "0.050000", 2023-01-24T19:28:08.108 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_max_osds": "4", 2023-01-24T19:28:08.108 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-24T19:28:08.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-24T19:28:08.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-24T19:28:08.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-24T19:28:08.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-24T19:28:08.109 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_interval": "300", 2023-01-24T19:28:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_max_keys": "100", 2023-01-24T19:28:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_timeout": "300", 2023-01-24T19:28:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_session_timeout": "300", 2023-01-24T19:28:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_smart_report_timeout": "5", 2023-01-24T19:28:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stat_smooth_intervals": "6", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_pool_min_size": "2", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_pool_size": "4", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-24T19:28:08.111 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_debug": "false", 2023-01-24T19:28:08.112 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-24T19:28:08.112 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-24T19:28:08.112 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_provider_kill_at": "0", 2023-01-24T19:28:08.112 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_requester_kill_at": "0", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_timeout": "60.000000", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_target_pg_per_osd": "100", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_tick_interval": "5", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_timecheck_interval": "300.000000", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-24T19:28:08.113 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_older_version_delay": "604800", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-24T19:28:08.114 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_misplaced": "false", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_older_version": "true", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-24T19:28:08.115 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-24T19:28:08.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-24T19:28:08.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-24T19:28:08.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-24T19:28:08.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-24T19:28:08.116 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-24T19:28:08.117 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-24T19:28:08.117 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-24T19:28:08.117 INFO:teuthology.orchestra.run.smithi062.stdout: "monmap": "", 2023-01-24T19:28:08.117 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_op_threads": "3", 2023-01-24T19:28:08.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-24T19:28:08.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_cm": "false", 2023-01-24T19:28:08.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_device_name": "", 2023-01-24T19:28:08.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_dscp": "96", 2023-01-24T19:28:08.118 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-24T19:28:08.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-24T19:28:08.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_local_gid": "", 2023-01-24T19:28:08.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-24T19:28:08.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_port_num": "1", 2023-01-24T19:28:08.119 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-24T19:28:08.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-24T19:28:08.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_roce_ver": "1", 2023-01-24T19:28:08.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_send_buffers": "1024", 2023-01-24T19:28:08.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_sl": "3", 2023-01-24T19:28:08.120 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_support_srq": "true", 2023-01-24T19:28:08.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_type": "ib", 2023-01-24T19:28:08.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_reap_threshold": "5", 2023-01-24T19:28:08.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_before_connect": "false", 2023-01-24T19:28:08.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_ipv4": "true", 2023-01-24T19:28:08.121 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_ipv6": "false", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_msgr1": "true", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_msgr2": "true", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_port_max": "7568", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_port_min": "6800", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_prefer_ipv4": "false", 2023-01-24T19:28:08.122 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_retry_count": "3", 2023-01-24T19:28:08.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_bind_retry_delay": "5", 2023-01-24T19:28:08.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_blackhole_client": "false", 2023-01-24T19:28:08.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_blackhole_mds": "false", 2023-01-24T19:28:08.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_blackhole_mgr": "false", 2023-01-24T19:28:08.123 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_blackhole_mon": "false", 2023-01-24T19:28:08.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_blackhole_osd": "false", 2023-01-24T19:28:08.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_client_mode": "crc secure", 2023-01-24T19:28:08.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_cluster_mode": "crc secure", 2023-01-24T19:28:08.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_cluster_type": "", 2023-01-24T19:28:08.124 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_compress_secure": "false", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_connection_idle_timeout": "900", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_connection_ready_timeout": "10", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_crc_data": "true", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_crc_header": "true", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_die_on_bad_msg": "false", 2023-01-24T19:28:08.125 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_die_on_bug": "true", 2023-01-24T19:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_die_on_old_message": "true", 2023-01-24T19:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_die_on_skipped_message": "false", 2023-01-24T19:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_die_on_unhandled_msg": "false", 2023-01-24T19:28:08.126 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-01-24T19:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_coremask": "0xF", 2023-01-24T19:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-01-24T19:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_devs_allowlist": "", 2023-01-24T19:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_enable_tso": "true", 2023-01-24T19:28:08.127 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-01-24T19:28:08.128 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-01-24T19:28:08.128 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_hugepages": "", 2023-01-24T19:28:08.128 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_hw_flow_control": "true", 2023-01-24T19:28:08.128 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_lro": "true", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_memory_channel": "4", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_pmd": "", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_port_id": "0", 2023-01-24T19:28:08.129 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-01-24T19:28:08.130 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dump_corrupt_message_level": "1", 2023-01-24T19:28:08.130 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_dump_on_send": "false", 2023-01-24T19:28:08.130 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_initial_backoff": "0.200000", 2023-01-24T19:28:08.130 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_inject_delay_max": "1.000000", 2023-01-24T19:28:08.130 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_inject_delay_probability": "0.005000", 2023-01-24T19:28:08.131 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_inject_delay_type": "osd", 2023-01-24T19:28:08.131 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_inject_internal_delays": "0.002000", 2023-01-24T19:28:08.131 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_inject_socket_failures": "2500", 2023-01-24T19:28:08.131 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_learn_addr_from_peer": "true", 2023-01-24T19:28:08.131 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_max_accept_failures": "4", 2023-01-24T19:28:08.132 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_max_backoff": "15.000000", 2023-01-24T19:28:08.132 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_mon_client_mode": "secure crc", 2023-01-24T19:28:08.132 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_mon_cluster_mode": "secure crc", 2023-01-24T19:28:08.132 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_mon_service_mode": "secure crc", 2023-01-24T19:28:08.132 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_osd_compress_min_size": "1024", 2023-01-24T19:28:08.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_osd_compress_mode": "none", 2023-01-24T19:28:08.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2023-01-24T19:28:08.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-01-24T19:28:08.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_pq_min_cost": "65536", 2023-01-24T19:28:08.133 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_public_type": "", 2023-01-24T19:28:08.134 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_service_mode": "crc secure", 2023-01-24T19:28:08.134 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_tcp_listen_backlog": "512", 2023-01-24T19:28:08.134 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_tcp_nodelay": "true", 2023-01-24T19:28:08.134 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-01-24T19:28:08.134 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_tcp_rcvbuf": "0", 2023-01-24T19:28:08.135 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_type": "async", 2023-01-24T19:28:08.135 INFO:teuthology.orchestra.run.smithi062.stdout: "no_config_file": "false", 2023-01-24T19:28:08.135 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_completion_locks_per_session": "32", 2023-01-24T19:28:08.135 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_debug_inject_relock_delay": "false", 2023-01-24T19:28:08.135 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_inflight_op_bytes": "104857600", 2023-01-24T19:28:08.136 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_inflight_ops": "1024", 2023-01-24T19:28:08.136 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_inject_no_watch_ping": "false", 2023-01-24T19:28:08.136 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-01-24T19:28:08.136 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_tick_interval": "5.000000", 2023-01-24T19:28:08.136 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter_timeout": "10.000000", 2023-01-24T19:28:08.137 INFO:teuthology.orchestra.run.smithi062.stdout: "objectstore_blackhole": "false", 2023-01-24T19:28:08.137 INFO:teuthology.orchestra.run.smithi062.stdout: "openssl_engine_opts": "", 2023-01-24T19:28:08.137 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_delay_time": "5.000000", 2023-01-24T19:28:08.137 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_hist_halflife": "1000", 2023-01-24T19:28:08.137 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_max_low_ops": "2", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_max_ops": "4", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_quantize_effort": "0.100000", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_agent_slop": "0.020000", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-01-24T19:28:08.138 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_allow_recovery_below_min_size": "true", 2023-01-24T19:28:08.139 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_asio_thread_count": "2", 2023-01-24T19:28:08.139 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_async_recovery_min_cost": "100", 2023-01-24T19:28:08.139 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_auto_mark_unfound_lost": "false", 2023-01-24T19:28:08.139 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backfill_retry_interval": "30.000000", 2023-01-24T19:28:08.139 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backfill_scan_max": "512", 2023-01-24T19:28:08.140 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backfill_scan_min": "64", 2023-01-24T19:28:08.140 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backoff_on_degraded": "false", 2023-01-24T19:28:08.140 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backoff_on_peering": "false", 2023-01-24T19:28:08.140 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_backoff_on_unfound": "true", 2023-01-24T19:28:08.140 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_beacon_report_interval": "300", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bench_duration": "30", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bench_max_block_size": "67108864", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bench_small_size_max_iops": "100", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_blkin_trace_all": "false", 2023-01-24T19:28:08.141 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_check_for_log_corruption": "false", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-01-24T19:28:08.142 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-01-24T19:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_class_update_on_start": "true", 2023-01-24T19:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_client_message_cap": "256", 2023-01-24T19:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_client_message_size_cap": "524288000", 2023-01-24T19:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_client_op_priority": "63", 2023-01-24T19:28:08.143 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_client_watch_timeout": "30", 2023-01-24T19:28:08.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_command_max_records": "256", 2023-01-24T19:28:08.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_compact_on_start": "false", 2023-01-24T19:28:08.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-01-24T19:28:08.144 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_crush_chooseleaf_type": "0", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_crush_initial_weight": "-1.000000", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_crush_update_on_start": "true", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_crush_update_weight_set": "true", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-01-24T19:28:08.145 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-01-24T19:28:08.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_drop_ping_duration": "0", 2023-01-24T19:28:08.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-01-24T19:28:08.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_feed_pullee": "-1", 2023-01-24T19:28:08.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-01-24T19:28:08.146 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-01-24T19:28:08.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-01-24T19:28:08.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_misdirected_ops": "false", 2023-01-24T19:28:08.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_no_acting_change": "false", 2023-01-24T19:28:08.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_no_purge_strays": "false", 2023-01-24T19:28:08.147 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_op_order": "false", 2023-01-24T19:28:08.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_pg_log_writeout": "false", 2023-01-24T19:28:08.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_pretend_recovery_active": "false", 2023-01-24T19:28:08.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-01-24T19:28:08.148 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-01-24T19:28:08.148 INFO:teuthology.orchestra.run.smithi0