2023-01-25T08:42:41.868 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T08:42:41.869 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121 --verbose --name yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi --owner scheduled_yuriw@teuthology --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} -- /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121/orig.config.yaml 2023-01-25T08:42:41.923 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T08:42:41.988 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121 branch: wip-yuri6-testing-2023-01-24-0746 description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy} email: yweinstein@radhat.com first_in_suite: false job_id: '7137121' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri6-testing-2023-01-24-0746 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr2: false ms inject socket failures: 5000 ms type: async osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd inject bad map crc probability: 0.1 osd max backfills: 3 osd max markdown count: 1000 osd max scrubs: 3 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate mon_bind_msgr2: false sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2023-01-24-0746 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri6-testing-2023-01-24-0746 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c targets: smithi006.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCefLXCGwe7hJJcQWjUvgCvKeQiI/mno2g4dps6sLEuBk4FgmbQB0Hd/fw+tPJg0LnxKeInVURif1yp2pe5vDOUB/nkwcCiN0POfXDGAHjMlDAil9Vx7Iui5UlX7Kc2qHMEGEQ6s3IEXLVXBk7mRItZ4pf/y8gixZTQeU3onF7qibPrRkLUWPmg3QIpgmA282RMdWZaQC4CLRyinDhH9+DfEpx4DEor7k5a/jmdVPYwsRJ6EI8WbP9unwJVKtW5Y0lXyyKf9NAe0zUPu54J6/TcqaDQANMMkROnRc4hhD24BEyFSpafeemPju/0tQVZ0p9yyPI2VvjFcFCE/uDycJRHM2CX4tLMWBVF0YUUTzxN28bRpXR17nK9vSVjK/3CqTNeV92iE3pJTR1yLmqn2i9Fa3rEwgF8Y5KWfDZoQ5nJB39lx8B8HhZ7Fu7KOO5iElT68FqLSACeNZH+VmBpwzGWavBPt+lmoe6xfr3lpfSLwfJwy9/Qbg6Xo2JzDeeJfwc= smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbBVW0ZM4KXHL0z52DF2cenLOEd3+t35UP8aqMMuL5YV2MGE1Q+WNeVWi6V7c9WXLg/meRERB3mS21t9dCdn7ZbrKGQCiN9oHFVRqlKdCijfHNkV5r1d/Y1eDFM7QTKoTMRNIVrrjCsW/48iNLOStilZxhUCsTpapeBwcYf7XnWGHY+aawuWeEaD7SmOuiRJRT8VuOBlDDXxVR2NRHf86P/xk6vFCP/VfmzkRCud8COElz0rh2mubJ0qv1Zx8qghzJnP1/AoMK+ggbORL93MDIfSib+WRV718ini5EHM+NBR6yIVfqey2zHE6VQrPWnA4ZZpPYRCA7iUHRXejIoKRh34PC2U/q6+q0YBo340u/T5hwd3fOsGYLt6vVNQV7QJLULvdHWWzlna6keh1VaeSDchfK2IxlkVOaiC5W3qHKXH9br2+uUpshjcU+k65hoLo2DlcSPlPh4DVShoVLuatgNYy/9NiEw0ZpJWjeVRfO+DIMcBc15yhRJCegZiuP+Qc= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache readproxy - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:27:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T08:42:41.989 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa; will attempt to use it 2023-01-25T08:42:41.990 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa/tasks 2023-01-25T08:42:41.990 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T08:42:41.992 INFO:teuthology.task.internal:Checking packages... 2023-01-25T08:42:42.019 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44e4095da12a8269eb6cfe163cbb0d00b1a6412c' 2023-01-25T08:42:42.019 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T08:42:42.020 INFO:teuthology.packaging:ref: None 2023-01-25T08:42:42.020 INFO:teuthology.packaging:tag: None 2023-01-25T08:42:42.021 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-01-24-0746 2023-01-25T08:42:42.021 INFO:teuthology.packaging:sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T08:42:42.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2023-01-24-0746 2023-01-25T08:42:42.354 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1986.g44e4095d 2023-01-25T08:42:42.354 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T08:42:42.362 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T08:42:42.362 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T08:42:42.408 INFO:teuthology.task.internal:Saving configuration 2023-01-25T08:42:42.427 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T08:42:42.434 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T08:42:42.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 08:37:51.971052', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGqMXjOsFhD1UuWTblRveXo/KsqY8dzJEWsDNc8MsAkpYy7Qi2zAnjChln7GwJyZrUn0CT84XuZhbmo0F/K3ps='} 2023-01-25T08:42:42.478 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 08:37:51.969755', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmMaCjLdBrhw8NgWhfEsM9xXPClcvbqrBhlZy3q4U9uQoO+aqw/n4AQc4Lfsu0AfkTLggkEDhUrGfChEXNxPmE='} 2023-01-25T08:42:42.479 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T08:42:42.485 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T08:42:42.485 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T08:42:42.485 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T08:42:42.561 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7dc658b670>, signals=[15]) 2023-01-25T08:42:42.562 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T08:42:42.569 INFO:teuthology.task.internal:Opening connections... 2023-01-25T08:42:42.569 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:42:42.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:42:42.643 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:42:42.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:42:42.715 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T08:42:42.722 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-01-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-01-25T08:42:42.744 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-01-25T08:42:42.803 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2023-01-25T08:42:42.803 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8" 2023-01-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2023-01-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2023-01-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8" 2023-01-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-01-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-01-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2023-01-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T08:42:42.806 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-01-25T08:42:42.823 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2023-01-25T08:42:42.847 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2023-01-25T08:42:42.847 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2023-01-25T08:42:42.908 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2023-01-25T08:42:42.909 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2023-01-25T08:42:42.909 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2023-01-25T08:42:42.909 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2023-01-25T08:42:42.909 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2023-01-25T08:42:42.910 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2023-01-25T08:42:42.910 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T08:42:42.910 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2023-01-25T08:42:42.910 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T08:42:42.910 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2023-01-25T08:42:42.911 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T08:42:42.911 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T08:42:42.911 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T08:42:42.911 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2023-01-25T08:42:42.928 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T08:42:42.941 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T08:42:42.954 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T08:42:42.955 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2023-01-25T08:42:42.957 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2023-01-25T08:42:42.978 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T08:42:42.984 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T08:42:42.985 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2023-01-25T08:42:43.017 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2023-01-25T08:42:43.046 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T08:42:43.103 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T08:42:43.122 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T08:42:43.123 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T08:42:43.124 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T08:42:43.125 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T08:42:43.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T08:42:43.144 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-01-25T08:42:43.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T08:42:43.146 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-01-25T08:42:43.204 INFO:teuthology.orchestra.run.smithi006.stdout:4.18.0-408.el8.x86_64 2023-01-25T08:42:43.205 INFO:teuthology.task.kernel:Running kernel on smithi006: 4.18.0-408.el8.x86_64 2023-01-25T08:42:43.205 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2023-01-25T08:42:43.207 INFO:teuthology.orchestra.run.smithi008.stdout:4.18.0-408.el8.x86_64 2023-01-25T08:42:43.207 INFO:teuthology.task.kernel:Running kernel on smithi008: 4.18.0-408.el8.x86_64 2023-01-25T08:42:43.207 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum install -y kernel 2023-01-25T08:42:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - AppStream 28 MB/s | 27 MB 00:00 2023-01-25T08:42:47.067 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - AppStream 26 MB/s | 27 MB 00:01 2023-01-25T08:42:52.937 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - BaseOS 43 MB/s | 27 MB 00:00 2023-01-25T08:42:53.501 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - BaseOS 28 MB/s | 27 MB 00:00 2023-01-25T08:42:57.486 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - Extras 154 kB/s | 18 kB 00:00 2023-01-25T08:42:57.905 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-25T08:42:58.000 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - Extras 152 kB/s | 18 kB 00:00 2023-01-25T08:42:58.411 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T08:42:59.187 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T08:42:59.677 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-01-25T08:43:01.955 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 13 MB 00:02 2023-01-25T08:43:03.677 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 13 MB 00:03 2023-01-25T08:43:05.480 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-01-25T08:43:07.120 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 452 kB/s | 24 kB 00:00 2023-01-25T08:43:08.778 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T08:43:08.778 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T08:43:08.963 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:43:08.964 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:43:08.964 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:43:08.965 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:43:08.965 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:43:08.965 INFO:teuthology.orchestra.run.smithi008.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T08:43:08.965 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T08:43:08.965 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T08:43:08.966 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:08.966 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:43:08.966 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:43:08.966 INFO:teuthology.orchestra.run.smithi008.stdout:Install 3 Packages 2023-01-25T08:43:08.966 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:08.967 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 85 M 2023-01-25T08:43:08.967 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 94 M 2023-01-25T08:43:08.968 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:43:09.231 INFO:teuthology.orchestra.run.smithi008.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 35 MB/s | 9.2 MB 00:00 2023-01-25T08:43:10.215 INFO:teuthology.orchestra.run.smithi008.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 33 MB/s | 41 MB 00:01 2023-01-25T08:43:10.392 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T08:43:10.393 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T08:43:10.562 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:43:10.563 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:43:10.563 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:43:10.563 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:43:10.563 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:43:10.564 INFO:teuthology.orchestra.run.smithi006.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T08:43:10.564 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T08:43:10.564 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T08:43:10.564 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:10.565 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:43:10.565 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:43:10.565 INFO:teuthology.orchestra.run.smithi006.stdout:Install 3 Packages 2023-01-25T08:43:10.565 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:10.566 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 85 M 2023-01-25T08:43:10.566 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 94 M 2023-01-25T08:43:10.566 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:43:10.711 INFO:teuthology.orchestra.run.smithi008.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 20 MB/s | 34 MB 00:01 2023-01-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:43:10.714 INFO:teuthology.orchestra.run.smithi008.stdout:Total 48 MB/s | 85 MB 00:01 2023-01-25T08:43:10.896 INFO:teuthology.orchestra.run.smithi006.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 28 MB/s | 9.2 MB 00:00 2023-01-25T08:43:11.375 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:43:11.737 INFO:teuthology.orchestra.run.smithi006.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 35 MB/s | 41 MB 00:01 2023-01-25T08:43:11.797 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:43:11.797 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:43:12.154 INFO:teuthology.orchestra.run.smithi006.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-01-25T08:43:12.157 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:43:12.157 INFO:teuthology.orchestra.run.smithi006.stdout:Total 53 MB/s | 85 MB 00:01 2023-01-25T08:43:12.814 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:43:13.235 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:43:13.235 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:43:13.798 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:43:13.840 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:43:15.233 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:43:15.277 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:43:16.752 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:43:17.318 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:18.153 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:43:18.859 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:21.343 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:21.784 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:22.819 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:23.217 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:25.335 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:25.581 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:26.776 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:27.020 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:49.051 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:49.077 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:49.523 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:49.524 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:49.524 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:50.130 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:50.130 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:50.131 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:43:50.131 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T08:43:50.131 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T08:43:50.131 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:50.131 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:43:50.242 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel --last | head -n 1 2023-01-25T08:43:50.444 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 08:43:25 AM UTC 2023-01-25T08:43:50.446 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T08:43:50.446 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T08:43:50.446 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T08:43:50.447 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T08:43:50.447 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-01-25T08:43:50.447 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi008.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T08:43:50.447 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel --last | head -n 1 2023-01-25T08:43:50.643 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 08:43:25 AM UTC 2023-01-25T08:43:50.645 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T08:43:50.645 DEBUG:teuthology.orchestra.run.smithi008:> sudo rpm -qi grub2-tools 2023-01-25T08:43:50.679 INFO:teuthology.orchestra.run.smithi008.stdout:Name : grub2-tools 2023-01-25T08:43:50.680 INFO:teuthology.orchestra.run.smithi008.stdout:Epoch : 1 2023-01-25T08:43:50.680 INFO:teuthology.orchestra.run.smithi008.stdout:Version : 2.02 2023-01-25T08:43:50.680 INFO:teuthology.orchestra.run.smithi008.stdout:Release : 129.el8 2023-01-25T08:43:50.680 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: x86_64 2023-01-25T08:43:50.681 INFO:teuthology.orchestra.run.smithi008.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T08:43:50.681 INFO:teuthology.orchestra.run.smithi008.stdout:Group : System Environment/Base 2023-01-25T08:43:50.681 INFO:teuthology.orchestra.run.smithi008.stdout:Size : 9265586 2023-01-25T08:43:50.681 INFO:teuthology.orchestra.run.smithi008.stdout:License : GPLv3+ 2023-01-25T08:43:50.681 INFO:teuthology.orchestra.run.smithi008.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T08:43:50.682 INFO:teuthology.orchestra.run.smithi008.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T08:43:50.682 INFO:teuthology.orchestra.run.smithi008.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T08:43:50.682 INFO:teuthology.orchestra.run.smithi008.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T08:43:50.682 INFO:teuthology.orchestra.run.smithi008.stdout:Relocations : (not relocatable) 2023-01-25T08:43:50.682 INFO:teuthology.orchestra.run.smithi008.stdout:Packager : CentOS BuildSystem 2023-01-25T08:43:50.683 INFO:teuthology.orchestra.run.smithi008.stdout:Vendor : CentOS 2023-01-25T08:43:50.683 INFO:teuthology.orchestra.run.smithi008.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T08:43:50.683 INFO:teuthology.orchestra.run.smithi008.stdout:Summary : Support tools for GRUB. 2023-01-25T08:43:50.683 INFO:teuthology.orchestra.run.smithi008.stdout:Description : 2023-01-25T08:43:50.683 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:50.684 INFO:teuthology.orchestra.run.smithi008.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T08:43:50.684 INFO:teuthology.orchestra.run.smithi008.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T08:43:50.684 INFO:teuthology.orchestra.run.smithi008.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T08:43:50.684 INFO:teuthology.orchestra.run.smithi008.stdout:hardware devices. 2023-01-25T08:43:50.684 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:43:50.685 INFO:teuthology.orchestra.run.smithi008.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T08:43:50.686 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T08:43:50.687 INFO:teuthology.task.kernel:Updating grub on smithi008 to boot 4.18.0-448.el8.x86_64 2023-01-25T08:43:50.687 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T08:43:50.891 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:50.918 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:51.362 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:51.363 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T08:43:51.363 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T08:43:51.988 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T08:43:51.988 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:51.988 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:43:51.989 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T08:43:51.989 INFO:teuthology.orchestra.run.smithi006.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T08:43:51.989 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:51.989 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:43:52.065 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2023-01-25T08:43:52.099 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel --last | head -n 1 2023-01-25T08:43:52.313 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 08:43:26 AM UTC 2023-01-25T08:43:52.314 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T08:43:52.315 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T08:43:52.315 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T08:43:52.315 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T08:43:52.315 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-25T08:43:52.316 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T08:43:52.316 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel --last | head -n 1 2023-01-25T08:43:52.512 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 08:43:26 AM UTC 2023-01-25T08:43:52.513 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T08:43:52.514 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2023-01-25T08:43:52.549 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2023-01-25T08:43:52.550 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2023-01-25T08:43:52.550 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.02 2023-01-25T08:43:52.550 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 129.el8 2023-01-25T08:43:52.550 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2023-01-25T08:43:52.550 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T08:43:52.551 INFO:teuthology.orchestra.run.smithi006.stdout:Group : System Environment/Base 2023-01-25T08:43:52.551 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 9265586 2023-01-25T08:43:52.551 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2023-01-25T08:43:52.551 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T08:43:52.551 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T08:43:52.552 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T08:43:52.552 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T08:43:52.552 INFO:teuthology.orchestra.run.smithi006.stdout:Relocations : (not relocatable) 2023-01-25T08:43:52.552 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : CentOS BuildSystem 2023-01-25T08:43:52.552 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2023-01-25T08:43:52.553 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T08:43:52.553 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2023-01-25T08:43:52.553 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2023-01-25T08:43:52.553 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:52.553 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T08:43:52.554 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T08:43:52.554 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T08:43:52.554 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2023-01-25T08:43:52.554 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:43:52.555 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T08:43:52.555 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T08:43:52.556 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 4.18.0-448.el8.x86_64 2023-01-25T08:43:52.556 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T08:43:53.873 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2023-01-25T08:43:54.422 INFO:teuthology.orchestra.run.smithi008.stderr:done 2023-01-25T08:43:54.424 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2023-01-25T08:43:54.462 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.Yb0wjNHuR3 2023-01-25T08:43:54.462 DEBUG:teuthology.orchestra.run.smithi008:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Yb0wjNHuR3 2023-01-25T08:43:54.536 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.Yb0wjNHuR3 2023-01-25T08:43:54.696 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.Yb0wjNHuR3 is 6KB 2023-01-25T08:43:54.745 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.Yb0wjNHuR3 2023-01-25T08:43:54.762 DEBUG:teuthology.orchestra.run.smithi008:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T08:43:54.836 INFO:teuthology.orchestra.run.smithi008.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T08:43:54.837 INFO:teuthology.orchestra.run.smithi008.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T08:43:54.837 INFO:teuthology.orchestra.run.smithi008.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T08:43:54.837 INFO:teuthology.orchestra.run.smithi008.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T08:43:54.838 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T08:43:55.013 DEBUG:teuthology.orchestra.run.smithi008:> sudo shutdown -r now 2023-01-25T08:43:56.050 INFO:teuthology.orchestra.run.smithi006.stderr:done 2023-01-25T08:43:56.052 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-01-25T08:43:56.081 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.ryCTJBXHHT 2023-01-25T08:43:56.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ryCTJBXHHT 2023-01-25T08:43:56.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.ryCTJBXHHT 2023-01-25T08:43:56.320 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.ryCTJBXHHT is 6KB 2023-01-25T08:43:56.368 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.ryCTJBXHHT 2023-01-25T08:43:56.386 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T08:43:56.451 INFO:teuthology.orchestra.run.smithi006.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T08:43:56.452 INFO:teuthology.orchestra.run.smithi006.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T08:43:56.452 INFO:teuthology.orchestra.run.smithi006.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T08:43:56.453 INFO:teuthology.orchestra.run.smithi006.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T08:43:56.454 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T08:43:56.628 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2023-01-25T08:44:25.044 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-01-25T08:44:25.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-01-25T08:44:25.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:44:26.632 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-25T08:44:26.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-01-25T08:44:26.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:44:43.500 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-01-25T08:44:45.032 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-01-25T08:45:13.528 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-01-25T08:45:13.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:45:15.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-01-25T08:45:15.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:45:30.446 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-01-25T08:45:30.692 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-01-25T08:45:30.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-01-25T08:45:30.814 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-01-25T08:45:30.814 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-01-25T08:45:30.833 INFO:teuthology.orchestra.run.smithi008.stdout:4.18.0-448.el8.x86_64 2023-01-25T08:45:30.833 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T08:45:30.833 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T08:45:30.833 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T08:45:30.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-01-25T08:45:30.971 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-25T08:45:30.971 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-01-25T08:45:31.025 INFO:teuthology.orchestra.run.smithi006.stdout:4.18.0-448.el8.x86_64 2023-01-25T08:45:31.025 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T08:45:31.026 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T08:45:31.026 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T08:45:31.834 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T08:45:31.834 DEBUG:teuthology.orchestra.run.smithi008:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T08:45:31.932 INFO:teuthology.orchestra.run.smithi008.stdout:ttyS1 2023-01-25T08:45:31.958 DEBUG:teuthology.parallel:result is None 2023-01-25T08:45:32.027 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T08:45:32.027 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T08:45:32.130 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2023-01-25T08:45:32.155 DEBUG:teuthology.parallel:result is None 2023-01-25T08:45:32.155 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T08:45:32.166 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T08:45:32.166 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T08:45:32.168 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T08:45:32.189 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T08:45:32.206 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T08:45:32.238 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T08:45:32.239 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T08:45:32.241 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T08:45:32.290 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T08:45:32.303 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T08:45:32.303 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T08:45:32.306 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T08:45:32.341 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T08:45:32.352 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T08:45:32.363 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T08:45:32.374 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T08:45:32.376 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T08:45:32.383 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T08:45:32.383 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T08:45:32.397 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T08:45:32.427 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T08:45:32.439 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T08:45:32.439 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T08:45:32.466 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T08:45:32.485 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T08:45:32.565 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T08:45:32.617 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T08:45:32.698 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T08:45:32.749 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:45:32.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T08:45:32.817 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T08:45:32.866 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T08:45:32.915 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T08:45:33.004 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T08:45:33.048 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:45:33.049 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T08:45:33.239 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2023-01-25T08:45:33.241 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2023-01-25T08:45:33.296 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T08:45:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T08:45:33.887 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T08:45:33.895 INFO:teuthology.task.internal:Starting timer... 2023-01-25T08:45:33.895 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T08:45:33.934 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T08:45:33.966 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T08:45:33.966 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2023-01-25T08:45:34.051 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2023-01-25T08:45:34.054 DEBUG:teuthology.orchestra.run.smithi008:> sudo service auditd rotate 2023-01-25T08:45:34.103 INFO:teuthology.orchestra.run.smithi008.stdout:Rotating logs: 2023-01-25T08:45:34.105 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T08:45:34.105 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2023-01-25T08:45:34.157 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2023-01-25T08:45:34.158 DEBUG:teuthology.orchestra.run.smithi008:> /usr/sbin/getenforce 2023-01-25T08:45:34.194 INFO:teuthology.orchestra.run.smithi008.stdout:Permissive 2023-01-25T08:45:34.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi008.front.sepia.ceph.com': 'permissive'} 2023-01-25T08:45:34.195 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T08:45:34.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T08:45:34.225 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T08:45:34.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T08:45:34.264 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T08:45:34.264 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2023-01-25T08:45:34.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo /usr/sbin/setenforce permissive 2023-01-25T08:45:34.515 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T08:45:34.524 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T08:45:34.525 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T08:45:34.540 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T08:45:34.542 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi008.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T08:52:13.318 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi008.front.sepia.ceph.com')] 2023-01-25T08:52:13.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-01-25T08:52:13.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:52:13.391 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-01-25T08:52:13.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-01-25T08:52:13.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T08:52:13.529 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-01-25T08:52:13.599 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T08:52:13.609 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T08:52:13.609 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T08:52:13.610 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T08:52:13.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T08:52:13.613 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T08:52:13.644 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T08:52:13.659 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T08:52:13.672 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T08:52:13.687 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T08:52:13.721 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2023-01-25T08:52:13.733 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2023-01-25T08:52:13.748 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T08:52:13.762 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T08:52:13.764 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: ntpd: command not found 2023-01-25T08:52:13.776 INFO:teuthology.orchestra.run.smithi008.stdout:506 Cannot talk to daemon 2023-01-25T08:52:13.792 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T08:52:13.806 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T08:52:13.832 INFO:teuthology.orchestra.run.smithi006.stderr:bash: ntpq: command not found 2023-01-25T08:52:13.836 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T08:52:13.836 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-01-25T08:52:13.837 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.837 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.838 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.838 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.870 INFO:teuthology.orchestra.run.smithi008.stderr:bash: ntpq: command not found 2023-01-25T08:52:13.875 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T08:52:13.875 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2023-01-25T08:52:13.875 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.875 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.876 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.876 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T08:52:13.876 INFO:teuthology.run_tasks:Running task install... 2023-01-25T08:52:13.885 DEBUG:teuthology.task.install:project ceph 2023-01-25T08:52:13.886 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T08:52:13.886 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'} 2023-01-25T08:52:13.887 INFO:teuthology.task.install:Using flavor: default 2023-01-25T08:52:13.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T08:52:13.893 INFO:teuthology.task.install:extra packages: [] 2023-01-25T08:52:13.894 DEBUG:teuthology.orchestra.run.smithi006:> sudo lsb_release -is 2023-01-25T08:52:13.932 INFO:teuthology.orchestra.run.smithi006.stdout:CentOSStream 2023-01-25T08:52:13.933 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:52:13.933 DEBUG:teuthology.orchestra.run.smithi008:> sudo lsb_release -is 2023-01-25T08:52:13.934 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T08:52:13.935 DEBUG:teuthology.orchestra.run.smithi006:> sudo lsb_release -is 2023-01-25T08:52:13.973 INFO:teuthology.orchestra.run.smithi008.stdout:CentOSStream 2023-01-25T08:52:13.974 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:52:13.974 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T08:52:13.975 DEBUG:teuthology.orchestra.run.smithi008:> sudo lsb_release -is 2023-01-25T08:52:14.012 INFO:teuthology.orchestra.run.smithi006.stdout:CentOSStream 2023-01-25T08:52:14.013 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:52:14.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T08:52:14.052 INFO:teuthology.orchestra.run.smithi008.stdout:CentOSStream 2023-01-25T08:52:14.054 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:52:14.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T08:52:14.322 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T08:52:14.322 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T08:52:14.372 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T08:52:14.373 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T08:52:14.626 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T08:52:14.627 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:52:14.627 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T08:52:14.629 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T08:52:14.630 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:52:14.630 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T08:52:14.660 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T08:52:14.660 DEBUG:teuthology.orchestra.run.smithi008:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T08:52:14.661 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T08:52:14.661 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T08:52:14.731 DEBUG:teuthology.orchestra.run.smithi008:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T08:52:14.733 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T08:52:14.812 DEBUG:teuthology.orchestra.run.smithi008:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T08:52:14.813 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T08:52:14.880 INFO:teuthology.orchestra.run.smithi008.stdout:check_obsoletes = 1 2023-01-25T08:52:14.881 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum clean all 2023-01-25T08:52:14.882 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2023-01-25T08:52:14.883 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2023-01-25T08:52:15.150 INFO:teuthology.orchestra.run.smithi008.stdout:61 files removed 2023-01-25T08:52:15.156 INFO:teuthology.orchestra.run.smithi006.stdout:61 files removed 2023-01-25T08:52:15.179 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-radosgw 2023-01-25T08:52:15.185 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw 2023-01-25T08:52:16.280 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 120 kB/s | 79 kB 00:00 2023-01-25T08:52:16.283 INFO:teuthology.orchestra.run.smithi008.stdout:ceph packages for x86_64 117 kB/s | 79 kB 00:00 2023-01-25T08:52:16.897 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 16 kB/s | 9.3 kB 00:00 2023-01-25T08:52:16.900 INFO:teuthology.orchestra.run.smithi008.stdout:ceph noarch packages 16 kB/s | 9.3 kB 00:00 2023-01-25T08:52:17.525 INFO:teuthology.orchestra.run.smithi008.stdout:ceph source packages 2.5 kB/s | 1.5 kB 00:00 2023-01-25T08:52:17.540 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 2.4 kB/s | 1.5 kB 00:00 2023-01-25T08:52:18.344 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - AppStream 35 MB/s | 27 MB 00:00 2023-01-25T08:52:18.484 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - AppStream 30 MB/s | 27 MB 00:00 2023-01-25T08:52:24.312 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - BaseOS 39 MB/s | 27 MB 00:00 2023-01-25T08:52:24.576 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - BaseOS 40 MB/s | 27 MB 00:00 2023-01-25T08:52:28.927 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - Extras 143 kB/s | 18 kB 00:00 2023-01-25T08:52:29.108 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - Extras 158 kB/s | 18 kB 00:00 2023-01-25T08:52:29.371 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS-8 - PowerTools 19 MB/s | 5.6 MB 00:00 2023-01-25T08:52:29.510 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-8 - PowerTools 22 MB/s | 5.6 MB 00:00 2023-01-25T08:52:30.654 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-01-25T08:52:30.826 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-01-25T08:52:35.704 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 13 MB 00:04 2023-01-25T08:52:35.793 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 13 MB 00:04 2023-01-25T08:52:39.171 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 396 kB/s | 24 kB 00:00 2023-01-25T08:52:39.360 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 541 kB/s | 24 kB 00:00 2023-01-25T08:52:42.865 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:52:42.868 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:52:42.868 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:52:42.868 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:52:42.868 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:52:42.869 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T08:52:42.869 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrading: 2023-01-25T08:52:42.869 INFO:teuthology.orchestra.run.smithi008.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T08:52:42.869 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T08:52:42.869 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:52:42.870 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T08:52:42.870 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T08:52:42.870 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T08:52:42.870 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T08:52:42.870 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T08:52:42.871 INFO:teuthology.orchestra.run.smithi008.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T08:52:42.871 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T08:52:42.871 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T08:52:42.871 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T08:52:42.871 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T08:52:42.872 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T08:52:42.872 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T08:52:42.872 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse 2023-01-25T08:52:42.872 INFO:teuthology.orchestra.run.smithi008.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T08:52:42.872 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T08:52:42.873 INFO:teuthology.orchestra.run.smithi008.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T08:52:42.874 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:52:42.874 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:52:42.874 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:52:42.874 INFO:teuthology.orchestra.run.smithi008.stdout:Install 21 Packages 2023-01-25T08:52:42.874 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrade 2 Packages 2023-01-25T08:52:42.875 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:52:42.875 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 58 M 2023-01-25T08:52:42.875 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:52:43.110 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:52:43.113 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:52:43.113 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:52:43.113 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:52:43.114 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:52:43.114 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T08:52:43.114 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2023-01-25T08:52:43.114 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T08:52:43.114 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T08:52:43.115 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:52:43.115 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T08:52:43.115 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T08:52:43.115 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T08:52:43.115 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T08:52:43.116 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T08:52:43.116 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T08:52:43.116 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T08:52:43.116 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T08:52:43.116 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T08:52:43.117 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T08:52:43.117 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T08:52:43.117 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T08:52:43.117 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse 2023-01-25T08:52:43.118 INFO:teuthology.orchestra.run.smithi006.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T08:52:43.118 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T08:52:43.118 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T08:52:43.118 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T08:52:43.118 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T08:52:43.119 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T08:52:43.119 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T08:52:43.119 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T08:52:43.119 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:52:43.119 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:52:43.120 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:52:43.120 INFO:teuthology.orchestra.run.smithi006.stdout:Install 21 Packages 2023-01-25T08:52:43.120 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 2 Packages 2023-01-25T08:52:43.120 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:52:43.121 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 58 M 2023-01-25T08:52:43.121 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:52:44.645 INFO:teuthology.orchestra.run.smithi006.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 3.2 MB/s | 4.8 MB 00:01 2023-01-25T08:52:44.672 INFO:teuthology.orchestra.run.smithi008.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 2.7 MB/s | 4.8 MB 00:01 2023-01-25T08:52:44.729 INFO:teuthology.orchestra.run.smithi006.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 292 kB/s | 24 kB 00:00 2023-01-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi008.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 325 kB/s | 24 kB 00:00 2023-01-25T08:52:45.062 INFO:teuthology.orchestra.run.smithi006.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 2.2 MB/s | 767 kB 00:00 2023-01-25T08:52:45.164 INFO:teuthology.orchestra.run.smithi008.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 1.8 MB/s | 767 kB 00:00 2023-01-25T08:52:45.196 INFO:teuthology.orchestra.run.smithi006.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 3.5 MB/s | 474 kB 00:00 2023-01-25T08:52:45.315 INFO:teuthology.orchestra.run.smithi008.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 3.1 MB/s | 474 kB 00:00 2023-01-25T08:52:46.429 INFO:teuthology.orchestra.run.smithi006.stdout:(5/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8. 3.9 MB/s | 13 MB 00:03 2023-01-25T08:52:46.513 INFO:teuthology.orchestra.run.smithi006.stdout:(6/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 4.9 MB/s | 6.5 MB 00:01 2023-01-25T08:52:46.538 INFO:teuthology.orchestra.run.smithi006.stdout:(7/23): python3-ceph-argparse-18.0.0-1986.g44e4 411 kB/s | 45 kB 00:00 2023-01-25T08:52:46.672 INFO:teuthology.orchestra.run.smithi006.stdout:(8/23): python3-ceph-common-18.0.0-1986.g44e409 725 kB/s | 115 kB 00:00 2023-01-25T08:52:46.697 INFO:teuthology.orchestra.run.smithi006.stdout:(9/23): python3-cephfs-18.0.0-1986.g44e4095d.el 1.2 MB/s | 190 kB 00:00 2023-01-25T08:52:46.789 INFO:teuthology.orchestra.run.smithi006.stdout:(10/23): python3-rados-18.0.0-1986.g44e4095d.el 3.1 MB/s | 372 kB 00:00 2023-01-25T08:52:46.864 INFO:teuthology.orchestra.run.smithi006.stdout:(11/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.4 MB/s | 106 kB 00:00 2023-01-25T08:52:46.898 INFO:teuthology.orchestra.run.smithi006.stdout:(12/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 1.8 MB/s | 364 kB 00:00 2023-01-25T08:52:46.957 INFO:teuthology.orchestra.run.smithi006.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-01-25T08:52:46.982 INFO:teuthology.orchestra.run.smithi006.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-01-25T08:52:47.007 INFO:teuthology.orchestra.run.smithi006.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2023-01-25T08:52:47.032 INFO:teuthology.orchestra.run.smithi006.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 934 kB/s | 47 kB 00:00 2023-01-25T08:52:47.266 INFO:teuthology.orchestra.run.smithi006.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 252 kB/s | 59 kB 00:00 2023-01-25T08:52:47.356 INFO:teuthology.orchestra.run.smithi008.stdout:(5/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 3.2 MB/s | 6.5 MB 00:02 2023-01-25T08:52:47.392 INFO:teuthology.orchestra.run.smithi006.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 600 kB/s | 75 kB 00:00 2023-01-25T08:52:47.423 INFO:teuthology.orchestra.run.smithi008.stdout:(6/23): python3-ceph-argparse-18.0.0-1986.g44e4 671 kB/s | 45 kB 00:00 2023-01-25T08:52:47.499 INFO:teuthology.orchestra.run.smithi008.stdout:(7/23): python3-ceph-common-18.0.0-1986.g44e409 1.5 MB/s | 115 kB 00:00 2023-01-25T08:52:47.574 INFO:teuthology.orchestra.run.smithi008.stdout:(8/23): python3-cephfs-18.0.0-1986.g44e4095d.el 2.5 MB/s | 190 kB 00:00 2023-01-25T08:52:47.692 INFO:teuthology.orchestra.run.smithi006.stdout:(19/23): ceph-common-18.0.0-1986.g44e4095d.el8. 4.7 MB/s | 21 MB 00:04 2023-01-25T08:52:47.725 INFO:teuthology.orchestra.run.smithi006.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 427 kB/s | 306 kB 00:00 2023-01-25T08:52:47.749 INFO:teuthology.orchestra.run.smithi008.stdout:(9/23): python3-rados-18.0.0-1986.g44e4095d.el8 2.1 MB/s | 372 kB 00:00 2023-01-25T08:52:47.966 INFO:teuthology.orchestra.run.smithi008.stdout:(10/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 1.6 MB/s | 364 kB 00:00 2023-01-25T08:52:48.050 INFO:teuthology.orchestra.run.smithi008.stdout:(11/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.3 MB/s | 106 kB 00:00 2023-01-25T08:52:48.109 INFO:teuthology.orchestra.run.smithi008.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-01-25T08:52:48.142 INFO:teuthology.orchestra.run.smithi008.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.7 MB/s | 259 kB 00:00 2023-01-25T08:52:48.159 INFO:teuthology.orchestra.run.smithi008.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 2.6 MB/s | 44 kB 00:00 2023-01-25T08:52:48.184 INFO:teuthology.orchestra.run.smithi008.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-25T08:52:48.292 INFO:teuthology.orchestra.run.smithi006.stdout:(21/23): librados2-18.0.0-1986.g44e4095d.el8.x8 6.1 MB/s | 3.6 MB 00:00 2023-01-25T08:52:48.509 INFO:teuthology.orchestra.run.smithi006.stdout:(22/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 4.5 MB/s | 3.6 MB 00:00 2023-01-25T08:52:48.811 INFO:teuthology.orchestra.run.smithi008.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 488 kB/s | 306 kB 00:00 2023-01-25T08:52:48.876 INFO:teuthology.orchestra.run.smithi008.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 911 kB/s | 59 kB 00:00 2023-01-25T08:52:48.985 INFO:teuthology.orchestra.run.smithi008.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 692 kB/s | 75 kB 00:00 2023-01-25T08:52:49.001 INFO:teuthology.orchestra.run.smithi006.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.1 MB/s | 1.7 MB 00:01 2023-01-25T08:52:49.005 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:52:49.005 INFO:teuthology.orchestra.run.smithi006.stdout:Total 9.9 MB/s | 58 MB 00:05 2023-01-25T08:52:49.048 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:52:49.098 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:52:49.098 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:52:49.627 INFO:teuthology.orchestra.run.smithi008.stdout:(19/23): ceph-common-18.0.0-1986.g44e4095d.el8. 3.2 MB/s | 21 MB 00:06 2023-01-25T08:52:49.748 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:52:49.752 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:52:50.219 INFO:teuthology.orchestra.run.smithi008.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 1.4 MB/s | 1.7 MB 00:01 2023-01-25T08:52:50.993 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:52:51.023 INFO:teuthology.orchestra.run.smithi008.stdout:(21/23): librados2-18.0.0-1986.g44e4095d.el8.x8 2.6 MB/s | 3.6 MB 00:01 2023-01-25T08:52:51.171 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T08:52:51.436 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T08:52:51.444 INFO:teuthology.orchestra.run.smithi008.stdout:(22/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 2.9 MB/s | 3.6 MB 00:01 2023-01-25T08:52:51.606 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T08:52:51.853 INFO:teuthology.orchestra.run.smithi008.stdout:(23/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8 1.4 MB/s | 13 MB 00:08 2023-01-25T08:52:51.856 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:52:51.857 INFO:teuthology.orchestra.run.smithi008.stdout:Total 6.5 MB/s | 58 MB 00:08 2023-01-25T08:52:51.898 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:52:51.948 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:52:51.948 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:52:52.042 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T08:52:52.248 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:52:52.345 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:52:52.582 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:52:52.585 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:52:52.624 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T08:52:52.814 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T08:52:53.231 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T08:52:53.398 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:52:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:52:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T08:52:53.781 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T08:52:53.852 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T08:52:53.926 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:52:54.219 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T08:52:54.478 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T08:52:54.536 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T08:52:54.706 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T08:52:54.777 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T08:52:54.939 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T08:52:55.017 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T08:52:55.139 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T08:52:55.229 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T08:52:55.347 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:52:55.447 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:52:55.749 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T08:52:55.939 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T08:52:56.099 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T08:52:56.272 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T08:52:56.346 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T08:52:56.358 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T08:52:56.531 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:52:56.547 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T08:52:56.592 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:52:56.734 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T08:52:56.784 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T08:52:56.931 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T08:52:56.939 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T08:52:57.143 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T08:52:57.368 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T08:52:57.780 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T08:52:57.852 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T08:52:58.035 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T08:52:58.982 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T08:52:59.180 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T08:52:59.239 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T08:52:59.623 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T08:52:59.822 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T08:53:00.056 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T08:53:00.650 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:00.855 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:01.440 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T08:53:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T08:53:02.005 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:03.767 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:04.022 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T08:53:04.935 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T08:53:05.022 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T08:53:05.230 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:21.785 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:21.785 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2023-01-25T08:53:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2023-01-25T08:53:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-01-25T08:53:21.786 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /run 2023-01-25T08:53:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2023-01-25T08:53:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2023-01-25T08:53:21.787 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2023-01-25T08:53:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2023-01-25T08:53:21.789 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-01-25T08:53:21.789 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:23.258 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T08:53:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T08:53:23.280 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T08:53:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T08:53:23.481 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T08:53:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T08:53:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T08:53:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:53:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T08:53:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T08:53:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:53:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T08:53:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T08:53:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T08:53:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T08:53:24.241 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T08:53:24.242 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T08:53:24.242 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T08:53:24.242 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T08:53:24.242 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T08:53:24.242 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:24.243 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T08:53:24.243 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T08:53:25.093 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:25.093 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /sys 2023-01-25T08:53:25.094 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /proc 2023-01-25T08:53:25.094 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2023-01-25T08:53:25.094 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /run 2023-01-25T08:53:25.094 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /mnt 2023-01-25T08:53:25.094 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /var/tmp 2023-01-25T08:53:25.095 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /home 2023-01-25T08:53:25.095 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /tmp 2023-01-25T08:53:25.095 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2023-01-25T08:53:25.095 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:25.159 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:25.159 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:25.160 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2023-01-25T08:53:25.160 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.160 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.160 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:53:25.160 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.161 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.161 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.162 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.162 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T08:53:25.163 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.163 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T08:53:25.164 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T08:53:25.164 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.164 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T08:53:25.165 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.165 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T08:53:25.165 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T08:53:25.166 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.166 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.167 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.167 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T08:53:25.167 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.168 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.168 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:25.168 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T08:53:25.169 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:25.169 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:53:25.469 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-test 2023-01-25T08:53:26.220 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:47 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:53:26.599 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T08:53:26.621 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T08:53:26.622 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T08:53:26.871 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T08:53:26.871 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:27.450 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:53:27.452 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:53:27.452 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:53:27.452 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:53:27.453 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:53:27.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T08:53:27.453 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:53:27.454 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T08:53:27.454 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T08:53:27.455 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T08:53:27.455 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T08:53:27.455 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T08:53:27.456 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:27.456 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:53:27.457 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:53:27.457 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2023-01-25T08:53:27.457 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:27.458 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 46 M 2023-01-25T08:53:27.459 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 204 M 2023-01-25T08:53:27.459 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:53:27.524 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-01-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.4 MB/s | 187 kB 00:00 2023-01-25T08:53:27.583 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2023-01-25T08:53:27.608 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.9 MB/s | 71 kB 00:00 2023-01-25T08:53:27.610 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:27.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T08:53:27.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T08:53:27.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T08:53:27.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T08:53:27.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T08:53:27.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T08:53:27.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T08:53:27.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T08:53:27.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T08:53:27.613 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T08:53:27.614 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T08:53:27.614 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T08:53:27.614 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T08:53:27.614 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T08:53:27.614 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T08:53:27.615 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T08:53:27.615 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T08:53:27.615 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T08:53:27.615 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T08:53:27.783 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 418 kB/s | 137 kB 00:00 2023-01-25T08:53:28.454 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T08:53:28.454 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:28.454 INFO:teuthology.orchestra.run.smithi008.stdout:Upgraded: 2023-01-25T08:53:28.455 INFO:teuthology.orchestra.run.smithi008.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.455 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.455 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:53:28.455 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.455 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.456 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.456 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.456 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T08:53:28.456 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.457 INFO:teuthology.orchestra.run.smithi008.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T08:53:28.457 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T08:53:28.457 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.457 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T08:53:28.457 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.458 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T08:53:28.458 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T08:53:28.458 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.458 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.458 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.459 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T08:53:28.459 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.459 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.459 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:28.459 INFO:teuthology.orchestra.run.smithi008.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T08:53:28.460 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:28.460 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:53:28.772 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-test 2023-01-25T08:53:29.534 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:00:50 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:53:30.778 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:53:30.779 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:53:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:53:30.780 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:53:30.780 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:53:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T08:53:30.781 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:53:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T08:53:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T08:53:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T08:53:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T08:53:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T08:53:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:30.784 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:53:30.784 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:53:30.785 INFO:teuthology.orchestra.run.smithi008.stdout:Install 6 Packages 2023-01-25T08:53:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:30.786 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 46 M 2023-01-25T08:53:30.786 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 204 M 2023-01-25T08:53:30.787 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:53:30.851 INFO:teuthology.orchestra.run.smithi008.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-01-25T08:53:30.875 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 13 MB/s | 45 MB 00:03 2023-01-25T08:53:30.878 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:53:30.878 INFO:teuthology.orchestra.run.smithi006.stdout:Total 13 MB/s | 46 MB 00:03 2023-01-25T08:53:30.893 INFO:teuthology.orchestra.run.smithi008.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 5.8 MB/s | 187 kB 00:00 2023-01-25T08:53:30.924 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:53:30.927 INFO:teuthology.orchestra.run.smithi008.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-01-25T08:53:30.947 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:53:30.947 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:53:30.952 INFO:teuthology.orchestra.run.smithi008.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.9 MB/s | 71 kB 00:00 2023-01-25T08:53:31.218 INFO:teuthology.orchestra.run.smithi008.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 315 kB/s | 137 kB 00:00 2023-01-25T08:53:31.257 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:53:31.261 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:53:31.730 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:53:31.948 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T08:53:32.150 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T08:53:32.434 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T08:53:32.513 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T08:53:32.722 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T08:53:32.868 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T08:53:37.765 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T08:53:38.093 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T08:53:38.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T08:53:38.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T08:53:38.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T08:53:38.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T08:53:38.094 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T08:53:38.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T08:53:38.822 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:38.823 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:53:38.823 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:38.825 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-6.el8.x86_64 2023-01-25T08:53:38.826 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:38.826 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T08:53:38.826 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T08:53:38.826 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T08:53:38.827 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:38.827 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:53:38.860 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph 2023-01-25T08:53:39.619 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:00 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:53:40.841 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:53:40.844 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2023-01-25T08:53:40.844 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:53:40.845 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2023-01-25T08:53:40.845 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:53:40.845 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T08:53:40.845 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:53:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T08:53:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T08:53:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T08:53:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T08:53:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T08:53:40.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T08:53:40.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T08:53:40.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T08:53:40.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T08:53:40.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T08:53:40.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T08:53:40.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T08:53:40.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T08:53:40.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T08:53:40.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T08:53:40.849 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T08:53:40.849 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T08:53:40.849 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T08:53:40.849 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T08:53:40.849 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T08:53:40.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T08:53:40.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T08:53:40.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T08:53:40.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T08:53:40.850 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T08:53:40.851 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T08:53:40.852 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:40.852 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:53:40.852 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================= 2023-01-25T08:53:40.852 INFO:teuthology.orchestra.run.smithi006.stdout:Install 32 Packages 2023-01-25T08:53:40.853 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:40.853 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 28 M 2023-01-25T08:53:40.853 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 107 M 2023-01-25T08:53:40.854 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:53:41.127 INFO:teuthology.orchestra.run.smithi006.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 23 kB/s | 6.4 kB 00:00 2023-01-25T08:53:41.486 INFO:teuthology.orchestra.run.smithi006.stdout:(2/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 2.4 MB/s | 1.5 MB 00:00 2023-01-25T08:53:41.536 INFO:teuthology.orchestra.run.smithi006.stdout:(3/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 3.3 MB/s | 2.3 MB 00:00 2023-01-25T08:53:41.677 INFO:teuthology.orchestra.run.smithi006.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 1.6 MB/s | 236 kB 00:00 2023-01-25T08:53:41.736 INFO:teuthology.orchestra.run.smithi006.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T08:53:41.761 INFO:teuthology.orchestra.run.smithi006.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T08:53:41.787 INFO:teuthology.orchestra.run.smithi006.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.1 MB/s | 103 kB 00:00 2023-01-25T08:53:41.812 INFO:teuthology.orchestra.run.smithi006.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-25T08:53:41.863 INFO:teuthology.orchestra.run.smithi006.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 9.0 MB/s | 457 kB 00:00 2023-01-25T08:53:41.888 INFO:teuthology.orchestra.run.smithi006.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.4 MB/s | 237 kB 00:00 2023-01-25T08:53:41.921 INFO:teuthology.orchestra.run.smithi006.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T08:53:41.980 INFO:teuthology.orchestra.run.smithi006.stdout:(12/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86 4.6 MB/s | 3.9 MB 00:00 2023-01-25T08:53:42.005 INFO:teuthology.orchestra.run.smithi006.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.3 MB/s | 109 kB 00:00 2023-01-25T08:53:42.230 INFO:teuthology.orchestra.run.smithi006.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 174 kB/s | 44 kB 00:00 2023-01-25T08:53:42.347 INFO:teuthology.orchestra.run.smithi006.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 541 kB/s | 185 kB 00:00 2023-01-25T08:53:42.439 INFO:teuthology.orchestra.run.smithi006.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 827 kB/s | 173 kB 00:00 2023-01-25T08:53:42.515 INFO:teuthology.orchestra.run.smithi006.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-01-25T08:53:42.590 INFO:teuthology.orchestra.run.smithi006.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 234 kB/s | 18 kB 00:00 2023-01-25T08:53:42.707 INFO:teuthology.orchestra.run.smithi006.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.0 MB/s | 384 kB 00:00 2023-01-25T08:53:42.724 INFO:teuthology.orchestra.run.smithi006.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 365 kB/s | 49 kB 00:00 2023-01-25T08:53:42.783 INFO:teuthology.orchestra.run.smithi006.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 782 kB/s | 59 kB 00:00 2023-01-25T08:53:42.866 INFO:teuthology.orchestra.run.smithi006.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2023-01-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi006.stdout:(23/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 11 MB/s | 17 MB 00:01 2023-01-25T08:53:43.075 INFO:teuthology.orchestra.run.smithi006.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 90 kB/s | 19 kB 00:00 2023-01-25T08:53:43.101 INFO:teuthology.orchestra.run.smithi006.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 752 kB/s | 283 kB 00:00 2023-01-25T08:53:43.159 INFO:teuthology.orchestra.run.smithi006.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 350 kB/s | 29 kB 00:00 2023-01-25T08:53:43.185 INFO:teuthology.orchestra.run.smithi006.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 329 kB/s | 27 kB 00:00 2023-01-25T08:53:43.260 INFO:teuthology.orchestra.run.smithi006.stdout:(28/32): python3-singledispatch-3.4.0.3-18.el8. 114 kB/s | 24 kB 00:00 2023-01-25T08:53:43.377 INFO:teuthology.orchestra.run.smithi006.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.1 MB/s | 241 kB 00:00 2023-01-25T08:53:43.468 INFO:teuthology.orchestra.run.smithi006.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 251 kB/s | 23 kB 00:00 2023-01-25T08:53:43.719 INFO:teuthology.orchestra.run.smithi006.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 470 kB/s | 251 kB 00:00 2023-01-25T08:53:43.744 INFO:teuthology.orchestra.run.smithi006.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 176 kB/s | 85 kB 00:00 2023-01-25T08:53:43.747 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:53:43.747 INFO:teuthology.orchestra.run.smithi006.stdout:Total 9.7 MB/s | 28 MB 00:02 2023-01-25T08:53:43.834 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:53:43.874 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:53:43.875 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:53:44.262 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:53:44.266 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:53:45.488 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:53:45.576 INFO:teuthology.orchestra.run.smithi008.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 3.0 MB/s | 45 MB 00:14 2023-01-25T08:53:45.579 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:53:45.580 INFO:teuthology.orchestra.run.smithi008.stdout:Total 3.1 MB/s | 46 MB 00:14 2023-01-25T08:53:45.623 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:53:45.646 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:53:45.646 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:53:45.750 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T08:53:45.959 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:53:45.963 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:53:46.150 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T08:53:46.401 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T08:53:46.436 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:53:46.690 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T08:53:46.814 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T08:53:46.879 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T08:53:47.014 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T08:53:47.054 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T08:53:47.174 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T08:53:47.265 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T08:53:47.278 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T08:53:47.448 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T08:53:47.470 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T08:53:47.634 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T08:53:47.674 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T08:53:47.936 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T08:53:48.175 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T08:53:48.396 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T08:53:48.825 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T08:53:49.112 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T08:53:49.333 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T08:53:49.569 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T08:53:49.801 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T08:53:50.090 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T08:53:50.406 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T08:53:50.628 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T08:53:50.863 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T08:53:51.115 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T08:53:51.337 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T08:53:51.616 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T08:53:51.985 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T08:53:52.339 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T08:53:52.344 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T08:53:52.663 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T08:53:52.953 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T08:53:52.975 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T08:53:53.523 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T08:53:53.523 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:53.524 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:53:53.524 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:53.524 INFO:teuthology.orchestra.run.smithi008.stdout: jq-1.6-6.el8.x86_64 2023-01-25T08:53:53.524 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:53.525 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T08:53:53.525 INFO:teuthology.orchestra.run.smithi008.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T08:53:53.525 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T08:53:53.525 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:53.525 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:53:53.747 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph 2023-01-25T08:53:54.500 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T08:53:54.504 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:15 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:53:54.706 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T08:53:55.132 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T08:53:55.324 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T08:53:55.550 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T08:53:55.727 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:53:55.728 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================= 2023-01-25T08:53:55.728 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:53:55.729 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================= 2023-01-25T08:53:55.729 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:53:55.729 INFO:teuthology.orchestra.run.smithi008.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T08:53:55.729 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:53:55.730 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T08:53:55.730 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T08:53:55.730 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T08:53:55.730 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T08:53:55.730 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T08:53:55.731 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T08:53:55.731 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T08:53:55.731 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T08:53:55.731 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T08:53:55.731 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T08:53:55.732 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T08:53:55.732 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T08:53:55.732 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T08:53:55.732 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T08:53:55.732 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T08:53:55.733 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T08:53:55.733 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T08:53:55.733 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T08:53:55.733 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T08:53:55.733 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T08:53:55.734 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T08:53:55.734 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T08:53:55.734 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T08:53:55.734 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T08:53:55.734 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T08:53:55.735 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T08:53:55.735 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T08:53:55.735 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T08:53:55.735 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T08:53:55.735 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================= 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout:Install 32 Packages 2023-01-25T08:53:55.736 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:53:55.737 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 28 M 2023-01-25T08:53:55.738 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 107 M 2023-01-25T08:53:55.738 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:53:55.748 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T08:53:55.768 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T08:53:55.906 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T08:53:55.970 INFO:teuthology.orchestra.run.smithi008.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 27 kB/s | 6.4 kB 00:00 2023-01-25T08:53:56.345 INFO:teuthology.orchestra.run.smithi008.stdout:(2/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 2.5 MB/s | 1.5 MB 00:00 2023-01-25T08:53:56.383 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T08:53:56.383 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T08:53:56.384 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T08:53:56.384 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T08:53:56.384 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T08:53:56.384 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T08:53:56.385 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T08:53:56.385 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T08:53:56.385 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T08:53:56.385 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T08:53:56.385 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T08:53:56.386 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T08:53:56.386 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T08:53:56.386 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T08:53:56.386 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T08:53:56.386 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T08:53:56.387 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T08:53:56.387 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T08:53:56.387 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T08:53:56.387 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T08:53:56.387 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T08:53:56.388 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T08:53:56.388 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T08:53:56.388 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T08:53:56.388 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T08:53:56.388 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T08:53:56.389 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T08:53:56.389 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T08:53:56.389 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T08:53:56.389 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T08:53:56.389 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T08:53:56.390 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T08:53:56.746 INFO:teuthology.orchestra.run.smithi008.stdout:(3/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86_ 5.0 MB/s | 3.9 MB 00:00 2023-01-25T08:53:56.837 INFO:teuthology.orchestra.run.smithi008.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 2.5 MB/s | 236 kB 00:00 2023-01-25T08:53:56.896 INFO:teuthology.orchestra.run.smithi008.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T08:53:56.921 INFO:teuthology.orchestra.run.smithi008.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T08:53:56.947 INFO:teuthology.orchestra.run.smithi008.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.1 MB/s | 103 kB 00:00 2023-01-25T08:53:56.963 INFO:teuthology.orchestra.run.smithi008.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 3.2 MB/s | 54 kB 00:00 2023-01-25T08:53:57.005 INFO:teuthology.orchestra.run.smithi008.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 11 MB/s | 457 kB 00:00 2023-01-25T08:53:57.039 INFO:teuthology.orchestra.run.smithi008.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 7.0 MB/s | 237 kB 00:00 2023-01-25T08:53:57.073 INFO:teuthology.orchestra.run.smithi008.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T08:53:57.098 INFO:teuthology.orchestra.run.smithi008.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T08:53:57.357 INFO:teuthology.orchestra.run.smithi008.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 169 kB/s | 44 kB 00:00 2023-01-25T08:53:57.632 INFO:teuthology.orchestra.run.smithi008.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 674 kB/s | 185 kB 00:00 2023-01-25T08:53:57.841 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T08:53:57.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:57.842 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:53:57.842 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:57.842 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:57.842 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:57.843 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:53:57.843 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:57.843 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:53:57.843 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T08:53:57.843 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T08:53:57.844 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T08:53:57.844 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T08:53:57.844 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T08:53:57.844 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T08:53:57.844 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T08:53:57.845 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T08:53:57.845 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T08:53:57.845 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T08:53:57.845 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T08:53:57.845 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T08:53:57.846 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T08:53:57.846 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T08:53:57.846 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T08:53:57.846 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T08:53:57.846 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T08:53:57.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T08:53:57.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T08:53:57.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T08:53:57.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T08:53:57.847 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T08:53:57.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T08:53:57.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T08:53:57.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T08:53:57.848 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T08:53:57.848 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:53:57.849 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:53:57.949 INFO:teuthology.orchestra.run.smithi008.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 545 kB/s | 173 kB 00:00 2023-01-25T08:53:58.065 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-base 2023-01-25T08:53:58.507 INFO:teuthology.orchestra.run.smithi008.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 689 kB/s | 384 kB 00:00 2023-01-25T08:53:58.599 INFO:teuthology.orchestra.run.smithi008.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-01-25T08:53:58.691 INFO:teuthology.orchestra.run.smithi008.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 192 kB/s | 18 kB 00:00 2023-01-25T08:53:58.758 INFO:teuthology.orchestra.run.smithi008.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 727 kB/s | 49 kB 00:00 2023-01-25T08:53:58.817 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:19 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:53:58.892 INFO:teuthology.orchestra.run.smithi008.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 440 kB/s | 59 kB 00:00 2023-01-25T08:53:59.201 INFO:teuthology.orchestra.run.smithi008.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 918 kB/s | 283 kB 00:00 2023-01-25T08:53:59.443 INFO:teuthology.orchestra.run.smithi008.stdout:(22/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 5.3 MB/s | 17 MB 00:03 2023-01-25T08:53:59.468 INFO:teuthology.orchestra.run.smithi008.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 58 kB/s | 16 kB 00:00 2023-01-25T08:53:59.544 INFO:teuthology.orchestra.run.smithi008.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 318 kB/s | 24 kB 00:00 2023-01-25T08:53:59.594 INFO:teuthology.orchestra.run.smithi008.stdout:(25/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86 603 kB/s | 2.3 MB 00:03 2023-01-25T08:53:59.619 INFO:teuthology.orchestra.run.smithi008.stdout:(26/32): python3-simplegeneric-0.8.1-17.el8.noa 107 kB/s | 19 kB 00:00 2023-01-25T08:53:59.644 INFO:teuthology.orchestra.run.smithi008.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2023-01-25T08:53:59.669 INFO:teuthology.orchestra.run.smithi008.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 365 kB/s | 27 kB 00:00 2023-01-25T08:53:59.917 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:54:00.029 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:00.030 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:54:00.031 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:00.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install cephadm 2023-01-25T08:54:00.254 INFO:teuthology.orchestra.run.smithi008.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 146 kB/s | 85 kB 00:00 2023-01-25T08:54:00.329 INFO:teuthology.orchestra.run.smithi008.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 339 kB/s | 241 kB 00:00 2023-01-25T08:54:00.354 INFO:teuthology.orchestra.run.smithi008.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 354 kB/s | 251 kB 00:00 2023-01-25T08:54:00.404 INFO:teuthology.orchestra.run.smithi008.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 153 kB/s | 23 kB 00:00 2023-01-25T08:54:00.407 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:00.408 INFO:teuthology.orchestra.run.smithi008.stdout:Total 6.0 MB/s | 28 MB 00:04 2023-01-25T08:54:00.492 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:00.532 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:00.533 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:00.863 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:21 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:00.925 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:00.928 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:02.092 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:02.093 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:02.093 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:02.093 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:02.094 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:54:02.095 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:02.095 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 82 k 2023-01-25T08:54:02.095 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 374 k 2023-01-25T08:54:02.095 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:02.233 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:02.515 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T08:54:02.517 INFO:teuthology.orchestra.run.smithi006.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 194 kB/s | 82 kB 00:00 2023-01-25T08:54:02.517 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:02.518 INFO:teuthology.orchestra.run.smithi006.stdout:Total 194 kB/s | 82 kB 00:00 2023-01-25T08:54:02.518 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:02.527 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:02.527 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:02.529 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:02.533 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:02.579 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:02.779 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T08:54:03.032 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T08:54:03.304 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T08:54:03.535 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T08:54:03.718 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T08:54:03.925 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T08:54:04.137 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T08:54:04.226 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:04.366 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T08:54:04.380 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:04.627 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:04.653 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T08:54:04.956 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T08:54:05.140 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:05.141 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:05.141 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:05.142 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:05.142 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:05.202 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T08:54:05.472 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T08:54:05.656 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T08:54:05.910 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T08:54:06.156 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T08:54:06.228 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:27 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:06.410 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T08:54:06.690 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T08:54:07.006 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T08:54:07.237 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T08:54:07.458 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T08:54:07.465 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:07.466 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:07.466 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2023-01-25T08:54:07.467 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:07.467 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T08:54:07.467 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:07.467 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:07.468 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:07.468 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:54:07.468 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:07.469 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 165 k 2023-01-25T08:54:07.469 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 503 k 2023-01-25T08:54:07.469 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:07.668 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T08:54:07.918 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T08:54:08.151 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T08:54:08.387 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T08:54:08.534 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 155 kB/s | 165 kB 00:01 2023-01-25T08:54:08.534 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:08.535 INFO:teuthology.orchestra.run.smithi006.stdout:Total 155 kB/s | 165 kB 00:01 2023-01-25T08:54:08.535 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:08.549 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:08.549 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:08.607 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:08.611 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:08.795 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T08:54:08.871 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:09.047 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:09.158 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T08:54:09.536 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:09.548 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T08:54:09.762 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T08:54:10.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:10.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:10.061 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:10.061 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:10.061 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:10.061 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:10.237 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr 2023-01-25T08:54:10.997 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:31 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:11.331 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T08:54:11.526 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T08:54:11.946 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T08:54:12.106 INFO:teuthology.orchestra.run.smithi006.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:54:12.113 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T08:54:12.216 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:12.217 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:54:12.218 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:12.301 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T08:54:12.342 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T08:54:12.514 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T08:54:12.532 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T08:54:12.645 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T08:54:13.066 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:13.121 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T08:54:13.121 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T08:54:13.121 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T08:54:13.121 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T08:54:13.122 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T08:54:13.122 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T08:54:13.122 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T08:54:13.122 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T08:54:13.122 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T08:54:13.123 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T08:54:13.123 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T08:54:13.123 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T08:54:13.123 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T08:54:13.124 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T08:54:13.124 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T08:54:13.124 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T08:54:13.124 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T08:54:13.124 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T08:54:13.125 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T08:54:13.125 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T08:54:13.125 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T08:54:13.125 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T08:54:13.125 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T08:54:13.126 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T08:54:13.126 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T08:54:13.126 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T08:54:13.127 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T08:54:13.127 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T08:54:13.128 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T08:54:13.128 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T08:54:13.128 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T08:54:13.128 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T08:54:14.132 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T08:54:14.132 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:14.133 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:54:14.133 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:14.133 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:14.133 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:14.133 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T08:54:14.134 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T08:54:14.135 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T08:54:14.135 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T08:54:14.135 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T08:54:14.135 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T08:54:14.135 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T08:54:14.136 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T08:54:14.136 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T08:54:14.136 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T08:54:14.136 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T08:54:14.136 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T08:54:14.137 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T08:54:14.137 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T08:54:14.137 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T08:54:14.137 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T08:54:14.137 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T08:54:14.138 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T08:54:14.138 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T08:54:14.138 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T08:54:14.138 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T08:54:14.138 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T08:54:14.139 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T08:54:14.139 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T08:54:14.139 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:14.139 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:14.276 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-base 2023-01-25T08:54:14.291 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:14.293 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:14.293 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:14.293 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:14.294 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T08:54:14.294 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:54:14.294 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T08:54:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T08:54:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T08:54:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T08:54:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T08:54:14.295 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:14.296 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:14.296 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:14.296 INFO:teuthology.orchestra.run.smithi006.stdout:Install 6 Packages 2023-01-25T08:54:14.296 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:14.297 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 2.0 M 2023-01-25T08:54:14.297 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 8.2 M 2023-01-25T08:54:14.297 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:14.620 INFO:teuthology.orchestra.run.smithi006.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 71 kB/s | 23 kB 00:00 2023-01-25T08:54:14.637 INFO:teuthology.orchestra.run.smithi006.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 43 kB/s | 15 kB 00:00 2023-01-25T08:54:14.695 INFO:teuthology.orchestra.run.smithi006.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 567 kB/s | 43 kB 00:00 2023-01-25T08:54:14.854 INFO:teuthology.orchestra.run.smithi006.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 155 kB/s | 34 kB 00:00 2023-01-25T08:54:15.029 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:15.121 INFO:teuthology.orchestra.run.smithi006.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 462 kB/s | 196 kB 00:00 2023-01-25T08:54:15.588 INFO:teuthology.orchestra.run.smithi006.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 1.3 MB/s | 1.7 MB 00:01 2023-01-25T08:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:15.591 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2023-01-25T08:54:15.603 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:15.613 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:15.613 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:15.707 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:15.710 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:15.994 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:16.136 INFO:teuthology.orchestra.run.smithi008.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:54:16.221 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T08:54:16.246 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:16.247 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:54:16.248 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:16.335 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install cephadm 2023-01-25T08:54:16.415 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T08:54:16.607 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T08:54:16.761 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T08:54:17.086 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:38 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T08:54:17.395 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T08:54:17.708 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T08:54:17.708 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T08:54:17.708 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T08:54:17.708 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T08:54:17.709 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T08:54:17.709 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T08:54:18.231 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T08:54:18.231 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:18.231 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:18.231 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:18.232 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:18.232 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:18.232 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T08:54:18.232 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T08:54:18.233 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T08:54:18.233 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:18.233 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:18.305 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:18.306 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:18.306 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:54:18.306 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:18.307 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:54:18.307 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T08:54:18.307 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:18.307 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:54:18.307 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:18.308 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:54:18.308 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:18.308 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 82 k 2023-01-25T08:54:18.309 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 374 k 2023-01-25T08:54:18.309 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:54:18.328 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T08:54:18.685 INFO:teuthology.orchestra.run.smithi008.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 218 kB/s | 82 kB 00:00 2023-01-25T08:54:18.686 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:18.686 INFO:teuthology.orchestra.run.smithi008.stdout:Total 217 kB/s | 82 kB 00:00 2023-01-25T08:54:18.686 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:18.694 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:18.694 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:18.696 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:18.700 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:18.750 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:19.091 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:20.330 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:20.331 INFO:teuthology.orchestra.run.smithi006.stdout:===================================================================================================== 2023-01-25T08:54:20.331 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:20.331 INFO:teuthology.orchestra.run.smithi006.stdout:===================================================================================================== 2023-01-25T08:54:20.332 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:20.332 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T08:54:20.332 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:54:20.332 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T08:54:20.332 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T08:54:20.333 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T08:54:20.333 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T08:54:20.333 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T08:54:20.333 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T08:54:20.333 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T08:54:20.334 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:20.334 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:20.334 INFO:teuthology.orchestra.run.smithi006.stdout:===================================================================================================== 2023-01-25T08:54:20.334 INFO:teuthology.orchestra.run.smithi006.stdout:Install 8 Packages 2023-01-25T08:54:20.334 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:20.335 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 35 M 2023-01-25T08:54:20.335 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 210 M 2023-01-25T08:54:20.336 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:20.386 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:20.495 INFO:teuthology.orchestra.run.smithi006.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 29 MB/s | 4.7 MB 00:00 2023-01-25T08:54:20.535 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:20.595 INFO:teuthology.orchestra.run.smithi006.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 38 MB/s | 3.7 MB 00:00 2023-01-25T08:54:20.629 INFO:teuthology.orchestra.run.smithi006.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.7 MB/s | 225 kB 00:00 2023-01-25T08:54:20.729 INFO:teuthology.orchestra.run.smithi006.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.9 MB 00:00 2023-01-25T08:54:20.779 INFO:teuthology.orchestra.run.smithi006.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-01-25T08:54:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:20.838 INFO:teuthology.orchestra.run.smithi006.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 2.9 MB/s | 172 kB 00:00 2023-01-25T08:54:21.021 INFO:teuthology.orchestra.run.smithi006.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 35 MB/s | 14 MB 00:00 2023-01-25T08:54:21.430 INFO:teuthology.orchestra.run.smithi006.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-198 6.8 MB/s | 7.4 MB 00:01 2023-01-25T08:54:21.432 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:21.432 INFO:teuthology.orchestra.run.smithi006.stdout:Total 32 MB/s | 35 MB 00:01 2023-01-25T08:54:21.671 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:21.691 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:22.128 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:22.131 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:22.155 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:54:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:22.156 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:22.726 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:22.861 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T08:54:22.994 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T08:54:23.161 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T08:54:23.324 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T08:54:23.800 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T08:54:23.987 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:24.098 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:45 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:24.868 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:25.261 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T08:54:25.329 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:25.329 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:25.329 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repo Size 2023-01-25T08:54:25.330 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:25.330 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:54:25.330 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T08:54:25.330 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:25.330 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:54:25.331 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:25.331 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:54:25.331 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:25.332 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 165 k 2023-01-25T08:54:25.332 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 503 k 2023-01-25T08:54:25.332 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:54:25.763 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 382 kB/s | 165 kB 00:00 2023-01-25T08:54:25.764 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:25.764 INFO:teuthology.orchestra.run.smithi008.stdout:Total 381 kB/s | 165 kB 00:00 2023-01-25T08:54:25.765 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:25.777 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:25.777 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:25.830 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:25.834 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:25.977 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:26.000 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T08:54:26.169 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:26.379 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T08:54:26.603 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T08:54:26.629 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:27.144 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T08:54:27.144 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:27.144 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:54:27.145 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:54:27.145 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:27.145 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:27.353 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr 2023-01-25T08:54:28.114 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:28.471 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T08:54:29.227 INFO:teuthology.orchestra.run.smithi008.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:54:29.338 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:29.340 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:54:29.340 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:29.425 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T08:54:29.806 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T08:54:30.019 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T08:54:30.183 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:51 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:30.363 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T08:54:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T08:54:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T08:54:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T08:54:30.364 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T08:54:30.365 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T08:54:30.365 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T08:54:30.952 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T08:54:30.953 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:30.953 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:30.953 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:30.953 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T08:54:30.954 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T08:54:30.954 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T08:54:30.954 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T08:54:30.954 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T08:54:30.954 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T08:54:30.955 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T08:54:30.955 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:30.955 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:31.068 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-rook 2023-01-25T08:54:31.404 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:31.405 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:54:31.405 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:31.405 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:54:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T08:54:31.406 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:54:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T08:54:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T08:54:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T08:54:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T08:54:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T08:54:31.408 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:31.408 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:54:31.408 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:54:31.408 INFO:teuthology.orchestra.run.smithi008.stdout:Install 6 Packages 2023-01-25T08:54:31.408 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:31.409 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 2.0 M 2023-01-25T08:54:31.410 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 8.2 M 2023-01-25T08:54:31.410 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:54:31.787 INFO:teuthology.orchestra.run.smithi008.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 61 kB/s | 23 kB 00:00 2023-01-25T08:54:31.817 INFO:teuthology.orchestra.run.smithi008.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 35 kB/s | 15 kB 00:00 2023-01-25T08:54:31.829 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:52 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:31.876 INFO:teuthology.orchestra.run.smithi008.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 479 kB/s | 43 kB 00:00 2023-01-25T08:54:32.310 INFO:teuthology.orchestra.run.smithi008.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 68 kB/s | 34 kB 00:00 2023-01-25T08:54:32.552 INFO:teuthology.orchestra.run.smithi008.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 291 kB/s | 196 kB 00:00 2023-01-25T08:54:32.793 INFO:teuthology.orchestra.run.smithi008.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 1.2 MB/s | 1.7 MB 00:01 2023-01-25T08:54:32.797 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:32.798 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2023-01-25T08:54:32.809 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:32.820 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:32.820 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:32.916 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:32.919 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:33.060 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:33.061 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2023-01-25T08:54:33.062 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:33.062 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2023-01-25T08:54:33.062 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:33.062 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T08:54:33.063 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:54:33.063 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T08:54:33.063 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T08:54:33.063 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T08:54:33.063 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T08:54:33.064 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T08:54:33.064 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T08:54:33.064 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T08:54:33.064 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T08:54:33.065 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T08:54:33.066 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T08:54:33.066 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T08:54:33.066 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T08:54:33.066 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:33.066 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:33.067 INFO:teuthology.orchestra.run.smithi006.stdout:======================================================================================== 2023-01-25T08:54:33.067 INFO:teuthology.orchestra.run.smithi006.stdout:Install 13 Packages 2023-01-25T08:54:33.067 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:33.068 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 1.9 M 2023-01-25T08:54:33.068 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 25 M 2023-01-25T08:54:33.068 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:33.145 INFO:teuthology.orchestra.run.smithi006.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 250 kB/s | 20 kB 00:00 2023-01-25T08:54:33.172 INFO:teuthology.orchestra.run.smithi006.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 252 kB/s | 27 kB 00:00 2023-01-25T08:54:33.230 INFO:teuthology.orchestra.run.smithi006.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.6 MB/s | 126 kB 00:00 2023-01-25T08:54:33.243 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:33.255 INFO:teuthology.orchestra.run.smithi006.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.3 MB/s | 110 kB 00:00 2023-01-25T08:54:33.281 INFO:teuthology.orchestra.run.smithi006.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T08:54:33.297 INFO:teuthology.orchestra.run.smithi006.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.0 MB/s | 43 kB 00:00 2023-01-25T08:54:33.451 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T08:54:33.464 INFO:teuthology.orchestra.run.smithi006.stdout:(7/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 119 kB/s | 47 kB 00:00 2023-01-25T08:54:33.855 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T08:54:34.021 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T08:54:34.142 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T08:54:34.223 INFO:teuthology.orchestra.run.smithi006.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 35 kB/s | 33 kB 00:00 2023-01-25T08:54:34.248 INFO:teuthology.orchestra.run.smithi006.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 17 kB/s | 16 kB 00:00 2023-01-25T08:54:34.348 INFO:teuthology.orchestra.run.smithi006.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 92 kB/s | 81 kB 00:00 2023-01-25T08:54:34.407 INFO:teuthology.orchestra.run.smithi006.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 387 kB/s | 61 kB 00:00 2023-01-25T08:54:34.466 INFO:teuthology.orchestra.run.smithi006.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 524 kB/s | 61 kB 00:00 2023-01-25T08:54:34.550 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T08:54:34.740 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T08:54:35.056 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T08:54:35.057 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T08:54:35.057 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T08:54:35.057 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T08:54:35.057 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T08:54:35.057 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T08:54:35.382 INFO:teuthology.orchestra.run.smithi006.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.0 MB/s | 1.2 MB 00:01 2023-01-25T08:54:35.384 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:35.385 INFO:teuthology.orchestra.run.smithi006.stdout:Total 850 kB/s | 1.9 MB 00:02 2023-01-25T08:54:35.444 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:35.461 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:35.461 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:35.568 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:35.571 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:35.574 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T08:54:35.574 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:35.574 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:54:35.574 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:35.575 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:35.575 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:35.575 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T08:54:35.575 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T08:54:35.576 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T08:54:35.576 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:35.576 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:35.691 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T08:54:36.439 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:36.449 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:57 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:36.749 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T08:54:36.955 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T08:54:37.198 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T08:54:37.459 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T08:54:37.642 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T08:54:37.671 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:37.677 INFO:teuthology.orchestra.run.smithi008.stdout:===================================================================================================== 2023-01-25T08:54:37.677 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:54:37.677 INFO:teuthology.orchestra.run.smithi008.stdout:===================================================================================================== 2023-01-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T08:54:37.679 INFO:teuthology.orchestra.run.smithi008.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T08:54:37.679 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T08:54:37.679 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T08:54:37.680 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T08:54:37.680 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T08:54:37.680 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:37.680 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:54:37.681 INFO:teuthology.orchestra.run.smithi008.stdout:===================================================================================================== 2023-01-25T08:54:37.681 INFO:teuthology.orchestra.run.smithi008.stdout:Install 8 Packages 2023-01-25T08:54:37.681 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:37.681 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 35 M 2023-01-25T08:54:37.681 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 210 M 2023-01-25T08:54:37.682 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:54:37.858 INFO:teuthology.orchestra.run.smithi008.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-01-25T08:54:37.929 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T08:54:37.967 INFO:teuthology.orchestra.run.smithi008.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 17 MB/s | 4.9 MB 00:00 2023-01-25T08:54:38.025 INFO:teuthology.orchestra.run.smithi008.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 22 MB/s | 3.7 MB 00:00 2023-01-25T08:54:38.051 INFO:teuthology.orchestra.run.smithi008.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 2.6 MB/s | 225 kB 00:00 2023-01-25T08:54:38.101 INFO:teuthology.orchestra.run.smithi008.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-01-25T08:54:38.164 INFO:teuthology.orchestra.run.smithi008.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 2.7 MB/s | 172 kB 00:00 2023-01-25T08:54:38.248 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T08:54:38.393 INFO:teuthology.orchestra.run.smithi008.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 37 MB/s | 14 MB 00:00 2023-01-25T08:54:38.526 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T08:54:39.677 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T08:54:39.950 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T08:54:40.192 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T08:54:40.405 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T08:54:40.560 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T08:54:40.887 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T08:54:40.888 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T08:54:40.889 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T08:54:40.889 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T08:54:40.889 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T08:54:40.890 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T08:54:40.890 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T08:54:40.890 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T08:54:40.891 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T08:54:40.891 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T08:54:40.891 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T08:54:40.892 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T08:54:40.892 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T08:54:41.489 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T08:54:41.489 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:41.489 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:41.489 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:41.490 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T08:54:41.490 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T08:54:41.490 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T08:54:41.490 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T08:54:41.490 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T08:54:41.491 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T08:54:41.491 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T08:54:41.491 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T08:54:41.491 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T08:54:41.492 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T08:54:41.492 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T08:54:41.492 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T08:54:41.492 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:41.492 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:41.603 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T08:54:41.929 INFO:teuthology.orchestra.run.smithi008.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-198 1.7 MB/s | 7.4 MB 00:04 2023-01-25T08:54:41.931 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:41.932 INFO:teuthology.orchestra.run.smithi008.stdout:Total 8.3 MB/s | 35 MB 00:04 2023-01-25T08:54:42.174 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:42.193 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:42.193 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:42.367 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:03 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:42.619 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:42.623 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:43.238 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:43.417 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T08:54:43.558 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T08:54:43.592 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:43.593 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:43.593 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:43.593 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:43.594 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:43.594 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T08:54:43.594 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:54:43.595 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T08:54:43.595 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T08:54:43.595 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T08:54:43.596 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T08:54:43.596 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:43.596 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:43.597 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:43.597 INFO:teuthology.orchestra.run.smithi006.stdout:Install 5 Packages 2023-01-25T08:54:43.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:43.598 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 6.0 M 2023-01-25T08:54:43.598 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 26 M 2023-01-25T08:54:43.598 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:43.692 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T08:54:43.742 INFO:teuthology.orchestra.run.smithi006.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 3.6 MB/s | 538 kB 00:00 2023-01-25T08:54:43.834 INFO:teuthology.orchestra.run.smithi006.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-01-25T08:54:43.968 INFO:teuthology.orchestra.run.smithi006.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 322 kB/s | 120 kB 00:00 2023-01-25T08:54:44.051 INFO:teuthology.orchestra.run.smithi006.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 278 kB/s | 60 kB 00:00 2023-01-25T08:54:44.321 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T08:54:44.435 INFO:teuthology.orchestra.run.smithi006.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 687 kB/s | 476 kB 00:00 2023-01-25T08:54:44.437 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:44.437 INFO:teuthology.orchestra.run.smithi006.stdout:Total 7.1 MB/s | 6.0 MB 00:00 2023-01-25T08:54:44.493 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:44.505 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:44.506 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:44.648 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:44.652 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:44.962 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:45.273 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T08:54:45.275 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:45.962 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:46.128 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T08:54:46.307 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T08:54:46.814 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T08:54:46.857 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T08:54:47.090 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T08:54:47.216 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T08:54:47.300 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T08:54:47.474 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T08:54:47.609 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T08:54:47.609 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T08:54:47.609 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T08:54:47.610 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T08:54:47.610 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T08:54:48.148 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T08:54:48.148 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:48.149 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:48.149 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:48.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T08:54:48.150 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T08:54:48.150 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T08:54:48.151 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T08:54:48.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:48.151 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:48.253 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-fuse 2023-01-25T08:54:49.007 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:10 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:49.351 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T08:54:50.233 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:50.234 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:50.234 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:50.234 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:50.235 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:50.235 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T08:54:50.235 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-01-25T08:54:50.235 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T08:54:50.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:50.236 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:50.236 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:50.236 INFO:teuthology.orchestra.run.smithi006.stdout:Install 2 Packages 2023-01-25T08:54:50.236 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:50.237 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 928 k 2023-01-25T08:54:50.237 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 2.9 M 2023-01-25T08:54:50.237 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:50.310 INFO:teuthology.orchestra.run.smithi006.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-25T08:54:50.647 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T08:54:51.185 INFO:teuthology.orchestra.run.smithi006.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 891 kB/s | 845 kB 00:00 2023-01-25T08:54:51.186 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:51.187 INFO:teuthology.orchestra.run.smithi006.stdout:Total 976 kB/s | 928 kB 00:00 2023-01-25T08:54:51.197 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:51.221 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:51.221 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:51.308 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T08:54:51.322 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:51.326 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:51.509 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:51.663 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T08:54:51.664 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T08:54:51.664 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T08:54:51.664 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T08:54:51.664 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T08:54:51.665 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T08:54:51.665 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T08:54:51.665 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T08:54:51.979 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T08:54:52.183 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T08:54:52.183 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T08:54:52.184 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:52.184 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:54:52.184 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:52.184 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T08:54:52.184 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T08:54:52.185 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T08:54:52.185 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T08:54:52.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T08:54:52.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T08:54:52.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T08:54:52.186 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:52.186 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:54:52.298 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-rook 2023-01-25T08:54:52.611 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T08:54:52.611 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T08:54:53.058 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:14 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T08:54:53.173 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:53.350 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-volume 2023-01-25T08:54:54.124 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:54.275 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:54:54.277 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================== 2023-01-25T08:54:54.277 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:54:54.277 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================== 2023-01-25T08:54:54.277 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:54:54.278 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T08:54:54.278 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:54:54.278 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T08:54:54.278 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T08:54:54.278 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T08:54:54.279 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T08:54:54.279 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T08:54:54.279 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T08:54:54.279 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T08:54:54.279 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T08:54:54.280 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T08:54:54.280 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T08:54:54.280 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T08:54:54.280 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T08:54:54.280 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:54.281 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:54:54.281 INFO:teuthology.orchestra.run.smithi008.stdout:======================================================================================== 2023-01-25T08:54:54.281 INFO:teuthology.orchestra.run.smithi008.stdout:Install 13 Packages 2023-01-25T08:54:54.281 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:54:54.282 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 1.9 M 2023-01-25T08:54:54.282 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 25 M 2023-01-25T08:54:54.282 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:54:54.360 INFO:teuthology.orchestra.run.smithi008.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 335 kB/s | 27 kB 00:00 2023-01-25T08:54:54.394 INFO:teuthology.orchestra.run.smithi008.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 177 kB/s | 20 kB 00:00 2023-01-25T08:54:54.419 INFO:teuthology.orchestra.run.smithi008.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.3 MB/s | 126 kB 00:00 2023-01-25T08:54:54.436 INFO:teuthology.orchestra.run.smithi008.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.6 MB/s | 110 kB 00:00 2023-01-25T08:54:54.461 INFO:teuthology.orchestra.run.smithi008.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.7 MB/s | 155 kB 00:00 2023-01-25T08:54:54.487 INFO:teuthology.orchestra.run.smithi008.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 847 kB/s | 43 kB 00:00 2023-01-25T08:54:54.653 INFO:teuthology.orchestra.run.smithi008.stdout:(7/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 127 kB/s | 47 kB 00:00 2023-01-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi008.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 89 kB/s | 16 kB 00:00 2023-01-25T08:54:54.696 INFO:teuthology.orchestra.run.smithi008.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 141 kB/s | 33 kB 00:00 2023-01-25T08:54:54.788 INFO:teuthology.orchestra.run.smithi008.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 607 kB/s | 81 kB 00:00 2023-01-25T08:54:54.913 INFO:teuthology.orchestra.run.smithi008.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 489 kB/s | 61 kB 00:00 2023-01-25T08:54:54.938 INFO:teuthology.orchestra.run.smithi008.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 254 kB/s | 61 kB 00:00 2023-01-25T08:54:55.346 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:54:55.348 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:55.348 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-01-25T08:54:55.349 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:55.349 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:54:55.349 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T08:54:55.350 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:55.350 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:54:55.351 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:54:55.351 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:54:55.351 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:55.352 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 263 k 2023-01-25T08:54:55.352 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 1.3 M 2023-01-25T08:54:55.352 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:54:55.680 INFO:teuthology.orchestra.run.smithi008.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.2 MB/s | 1.2 MB 00:01 2023-01-25T08:54:55.682 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:55.683 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.4 MB/s | 1.9 MB 00:01 2023-01-25T08:54:55.743 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:54:55.760 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:54:55.760 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:54:55.840 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 535 kB/s | 263 kB 00:00 2023-01-25T08:54:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:54:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:Total 533 kB/s | 263 kB 00:00 2023-01-25T08:54:55.841 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:54:55.853 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:54:55.853 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:54:55.867 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:54:55.872 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:54:55.964 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:54:55.967 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:54:56.160 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:54:56.395 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:54:56.449 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:56.662 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T08:54:56.885 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T08:54:56.901 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:57.152 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T08:54:57.381 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T08:54:57.453 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:54:57.453 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:57.453 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:54:57.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:54:57.454 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:54:57.454 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T08:54:57.841 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T08:54:58.146 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T08:54:58.431 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T08:54:58.713 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados-devel 2023-01-25T08:54:59.506 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:54:59.550 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T08:54:59.848 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T08:55:00.055 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T08:55:00.317 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T08:55:00.465 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T08:55:00.720 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:00.721 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:00.721 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2023-01-25T08:55:00.721 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:00.721 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:55:00.722 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T08:55:00.722 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:00.722 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:55:00.722 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:00.722 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:55:00.723 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:00.723 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 139 k 2023-01-25T08:55:00.723 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 512 k 2023-01-25T08:55:00.724 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:55:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T08:55:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T08:55:00.792 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T08:55:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T08:55:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T08:55:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T08:55:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T08:55:00.793 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T08:55:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T08:55:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T08:55:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T08:55:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T08:55:00.794 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T08:55:01.174 INFO:teuthology.orchestra.run.smithi006.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 308 kB/s | 139 kB 00:00 2023-01-25T08:55:01.175 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:01.175 INFO:teuthology.orchestra.run.smithi006.stdout:Total 307 kB/s | 139 kB 00:00 2023-01-25T08:55:01.175 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:55:01.181 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:55:01.181 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:55:01.239 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:55:01.242 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:55:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T08:55:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:01.373 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:01.373 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:55:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T08:55:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T08:55:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T08:55:01.374 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T08:55:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T08:55:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T08:55:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T08:55:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T08:55:01.375 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T08:55:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T08:55:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T08:55:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T08:55:01.376 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:01.377 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:01.389 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:55:01.501 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T08:55:01.547 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:01.858 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:02.268 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:02.357 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:02.358 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:55:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:02.358 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:02.358 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:02.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs2 2023-01-25T08:55:03.368 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:24 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:03.509 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:03.510 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:03.511 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:55:03.511 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:03.511 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:03.511 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T08:55:03.512 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:55:03.512 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T08:55:03.513 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T08:55:03.513 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T08:55:03.513 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T08:55:03.514 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:03.514 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:03.514 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:03.514 INFO:teuthology.orchestra.run.smithi008.stdout:Install 5 Packages 2023-01-25T08:55:03.515 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:03.516 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 6.0 M 2023-01-25T08:55:03.516 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 26 M 2023-01-25T08:55:03.516 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:03.730 INFO:teuthology.orchestra.run.smithi008.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 2.4 MB/s | 538 kB 00:00 2023-01-25T08:55:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-01-25T08:55:04.038 INFO:teuthology.orchestra.run.smithi008.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 228 kB/s | 120 kB 00:00 2023-01-25T08:55:04.479 INFO:teuthology.orchestra.run.smithi006.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:04.589 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:04.591 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:04.591 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:04.668 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install libcephfs-devel 2023-01-25T08:55:05.428 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:26 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:06.673 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:06.674 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:06.674 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2023-01-25T08:55:06.674 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:06.675 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:55:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T08:55:06.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:06.675 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:55:06.675 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:06.676 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:55:06.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:06.676 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 30 k 2023-01-25T08:55:06.676 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 137 k 2023-01-25T08:55:06.676 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:55:07.045 INFO:teuthology.orchestra.run.smithi006.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 82 kB/s | 30 kB 00:00 2023-01-25T08:55:07.045 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:07.046 INFO:teuthology.orchestra.run.smithi006.stdout:Total 82 kB/s | 30 kB 00:00 2023-01-25T08:55:07.046 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:55:07.049 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:55:07.050 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:55:07.076 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:55:07.079 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:55:07.167 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:55:07.380 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:07.674 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:07.964 INFO:teuthology.orchestra.run.smithi008.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 15 kB/s | 60 kB 00:04 2023-01-25T08:55:08.149 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:08.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:08.150 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:55:08.150 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:08.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:08.151 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:08.247 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librados2 2023-01-25T08:55:08.281 INFO:teuthology.orchestra.run.smithi008.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 105 kB/s | 476 kB 00:04 2023-01-25T08:55:08.283 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:08.283 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.2 MB/s | 6.0 MB 00:04 2023-01-25T08:55:08.340 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:08.352 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:08.353 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:08.502 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:08.505 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:08.782 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:09.004 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:30 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:09.078 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T08:55:10.077 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T08:55:10.134 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:10.248 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:10.249 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:10.250 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:10.434 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install librbd1 2023-01-25T08:55:10.501 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T08:55:10.769 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T08:55:10.955 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T08:55:11.204 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:32 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:11.256 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T08:55:11.256 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T08:55:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T08:55:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T08:55:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T08:55:11.808 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T08:55:11.808 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:11.808 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:11.808 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:55:11.809 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T08:55:11.809 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T08:55:11.809 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T08:55:11.809 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T08:55:11.809 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:11.810 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:11.915 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-fuse 2023-01-25T08:55:12.358 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:12.470 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:12.471 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:12.472 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:12.556 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rados 2023-01-25T08:55:12.667 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:13.315 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:34 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:13.897 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:13.897 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:13.897 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:55:13.898 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:13.898 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:13.898 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T08:55:13.898 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-01-25T08:55:13.898 INFO:teuthology.orchestra.run.smithi008.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T08:55:13.899 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:13.899 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:13.899 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:13.899 INFO:teuthology.orchestra.run.smithi008.stdout:Install 2 Packages 2023-01-25T08:55:13.899 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:13.900 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 928 k 2023-01-25T08:55:13.900 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 2.9 M 2023-01-25T08:55:13.900 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:13.971 INFO:teuthology.orchestra.run.smithi008.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-25T08:55:14.420 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:14.531 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:14.532 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:14.533 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:14.622 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rgw 2023-01-25T08:55:15.205 INFO:teuthology.orchestra.run.smithi008.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 647 kB/s | 845 kB 00:01 2023-01-25T08:55:15.206 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:15.207 INFO:teuthology.orchestra.run.smithi008.stdout:Total 709 kB/s | 928 kB 00:01 2023-01-25T08:55:15.219 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:15.244 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:15.245 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:15.351 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:15.355 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:15.374 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:36 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:15.539 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:15.808 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T08:55:16.004 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T08:55:16.434 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T08:55:16.434 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T08:55:16.492 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:16.603 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:16.604 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:16.604 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:16.699 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-cephfs 2023-01-25T08:55:17.239 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T08:55:17.239 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:17.240 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:17.240 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T08:55:17.240 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:17.240 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:17.448 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:18.015 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-volume 2023-01-25T08:55:18.575 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:18.688 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:18.689 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:18.690 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:18.781 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install python3-rbd 2023-01-25T08:55:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:39 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:19.534 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:40 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:20.017 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:20.018 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:20.018 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-01-25T08:55:20.018 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:20.018 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:20.019 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T08:55:20.019 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:20.019 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:20.020 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:20.020 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:20.020 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:20.020 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 263 k 2023-01-25T08:55:20.020 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 1.3 M 2023-01-25T08:55:20.021 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:20.653 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 415 kB/s | 263 kB 00:00 2023-01-25T08:55:20.654 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:20.654 INFO:teuthology.orchestra.run.smithi008.stdout:Total 414 kB/s | 263 kB 00:00 2023-01-25T08:55:20.654 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:20.665 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:20.666 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:20.688 INFO:teuthology.orchestra.run.smithi006.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:20.769 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:20.773 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:20.799 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:20.800 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-01-25T08:55:20.801 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:20.911 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-fuse 2023-01-25T08:55:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:55:21.682 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:55:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T08:55:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:22.235 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T08:55:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:22.236 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:22.425 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librados-devel 2023-01-25T08:55:22.910 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:22.911 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:22.911 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2023-01-25T08:55:22.912 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:22.912 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:55:22.912 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T08:55:22.913 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:22.913 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:55:22.914 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:22.914 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:55:22.914 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:22.915 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 94 k 2023-01-25T08:55:22.915 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 280 k 2023-01-25T08:55:22.915 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:55:23.186 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:44 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:23.377 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 203 kB/s | 94 kB 00:00 2023-01-25T08:55:23.378 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:23.378 INFO:teuthology.orchestra.run.smithi006.stdout:Total 202 kB/s | 94 kB 00:00 2023-01-25T08:55:23.378 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:55:23.390 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:55:23.391 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:55:23.448 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:55:23.451 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:55:23.566 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:55:23.786 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:24.139 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:24.405 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:24.406 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:24.407 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repo Size 2023-01-25T08:55:24.407 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:24.407 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:24.407 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T08:55:24.408 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:24.408 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:24.408 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:24.408 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:24.409 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:24.409 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 139 k 2023-01-25T08:55:24.409 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 512 k 2023-01-25T08:55:24.409 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:24.696 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:24.696 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:24.696 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:55:24.697 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:24.697 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:24.697 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:24.875 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-mirror 2023-01-25T08:55:24.885 INFO:teuthology.orchestra.run.smithi008.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 291 kB/s | 139 kB 00:00 2023-01-25T08:55:24.885 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:24.886 INFO:teuthology.orchestra.run.smithi008.stdout:Total 290 kB/s | 139 kB 00:00 2023-01-25T08:55:24.886 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:24.892 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:24.892 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:24.952 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:24.955 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:25.077 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:25.252 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:25.574 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:25.647 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:46 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:26.126 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:26.126 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:26.127 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:26.127 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:26.127 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:26.127 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:26.317 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install libcephfs2 2023-01-25T08:55:26.875 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:26.876 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:26.876 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repo Size 2023-01-25T08:55:26.876 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:26.876 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:55:26.877 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T08:55:26.877 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:26.877 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:55:26.877 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:26.878 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:55:26.878 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:26.878 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 3.5 M 2023-01-25T08:55:26.878 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 16 M 2023-01-25T08:55:26.879 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:55:27.093 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:48 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:28.211 INFO:teuthology.orchestra.run.smithi008.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:28.324 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:28.325 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:28.325 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:28.412 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install libcephfs-devel 2023-01-25T08:55:29.175 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:50 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:29.898 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 1.1 MB/s | 3.5 MB 00:03 2023-01-25T08:55:29.899 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:29.900 INFO:teuthology.orchestra.run.smithi006.stdout:Total 1.1 MB/s | 3.5 MB 00:03 2023-01-25T08:55:29.900 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:55:29.915 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:55:29.915 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:55:29.993 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:55:30.419 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:30.420 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:30.421 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repo Size 2023-01-25T08:55:30.421 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:30.421 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:30.421 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T08:55:30.422 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:30.422 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:30.422 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:30.422 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:30.422 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:30.423 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 30 k 2023-01-25T08:55:30.423 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 137 k 2023-01-25T08:55:30.423 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:30.469 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:55:30.654 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:30.708 INFO:teuthology.orchestra.run.smithi008.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 106 kB/s | 30 kB 00:00 2023-01-25T08:55:30.709 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:30.709 INFO:teuthology.orchestra.run.smithi008.stdout:Total 106 kB/s | 30 kB 00:00 2023-01-25T08:55:30.709 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:30.712 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:30.713 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:30.735 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:30.738 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:31.138 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:31.668 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:31.668 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:31.669 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:55:31.669 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:31.669 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:31.669 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:31.770 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:31.770 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:31.771 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:31.771 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:31.771 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:31.771 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:31.869 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librados2 2023-01-25T08:55:32.088 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install rbd-nbd 2023-01-25T08:55:32.643 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:53 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:32.847 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:02:53 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:33.755 INFO:teuthology.orchestra.run.smithi008.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:33.868 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:33.869 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:33.869 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:33.953 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librbd1 2023-01-25T08:55:34.066 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-01-25T08:55:34.066 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:34.067 INFO:teuthology.orchestra.run.smithi006.stdout: Package Architecture Version Repository Size 2023-01-25T08:55:34.067 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:34.067 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-01-25T08:55:34.067 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T08:55:34.067 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:34.068 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-01-25T08:55:34.068 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-01-25T08:55:34.068 INFO:teuthology.orchestra.run.smithi006.stdout:Install 1 Package 2023-01-25T08:55:34.068 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:34.069 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 180 k 2023-01-25T08:55:34.069 INFO:teuthology.orchestra.run.smithi006.stdout:Installed size: 568 k 2023-01-25T08:55:34.069 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-01-25T08:55:34.568 INFO:teuthology.orchestra.run.smithi006.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 359 kB/s | 180 kB 00:00 2023-01-25T08:55:34.569 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:34.569 INFO:teuthology.orchestra.run.smithi006.stdout:Total 358 kB/s | 180 kB 00:00 2023-01-25T08:55:34.570 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-01-25T08:55:34.582 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-01-25T08:55:34.582 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-01-25T08:55:34.640 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-01-25T08:55:34.644 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-01-25T08:55:34.709 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:55 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:34.766 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-01-25T08:55:34.968 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:35.269 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:35.777 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:35.778 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:35.778 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-01-25T08:55:35.778 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:35.778 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:55:35.779 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-01-25T08:55:35.824 INFO:teuthology.orchestra.run.smithi008.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:35.939 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:35.940 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:35.940 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:35.958 DEBUG:teuthology.parallel:result is None 2023-01-25T08:55:36.026 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rados 2023-01-25T08:55:36.785 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:57 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:37.900 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:38.014 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:38.016 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:38.016 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:38.103 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rgw 2023-01-25T08:55:38.856 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:59 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:39.974 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:40.086 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:40.087 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:40.087 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:40.178 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-cephfs 2023-01-25T08:55:40.944 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:01 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:42.051 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:42.161 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:42.163 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:42.163 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:42.247 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rbd 2023-01-25T08:55:43.000 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:03 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:44.115 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T08:55:44.227 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:44.228 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-01-25T08:55:44.229 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:44.321 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-fuse 2023-01-25T08:55:45.087 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:06 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:46.325 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:46.326 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:46.326 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2023-01-25T08:55:46.326 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:46.326 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:46.326 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T08:55:46.327 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:46.327 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:46.327 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:46.327 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:46.327 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:46.328 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 94 k 2023-01-25T08:55:46.328 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 280 k 2023-01-25T08:55:46.328 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:46.771 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 212 kB/s | 94 kB 00:00 2023-01-25T08:55:46.772 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:46.772 INFO:teuthology.orchestra.run.smithi008.stdout:Total 212 kB/s | 94 kB 00:00 2023-01-25T08:55:46.772 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:46.783 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:46.784 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:46.839 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:46.843 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:46.958 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:47.207 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:47.505 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:48.017 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:48.017 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:48.017 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:48.018 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:48.018 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:48.018 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:48.178 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-mirror 2023-01-25T08:55:48.949 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:50.217 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:50.218 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:50.218 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repo Size 2023-01-25T08:55:50.218 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:50.218 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:50.218 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T08:55:50.219 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:50.219 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:50.219 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:50.219 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:50.220 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:50.220 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 3.5 M 2023-01-25T08:55:50.220 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 16 M 2023-01-25T08:55:50.220 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:52.215 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 1.7 MB/s | 3.5 MB 00:01 2023-01-25T08:55:52.216 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:52.216 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.7 MB/s | 3.5 MB 00:01 2023-01-25T08:55:52.217 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:52.232 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:52.232 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:52.305 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:52.308 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:52.780 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:53.025 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:53.480 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:53.979 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:53.979 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:53.980 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:53.980 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:53.980 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:53.980 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:54.208 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-nbd 2023-01-25T08:55:54.967 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:15 ago on Wed 25 Jan 2023 08:52:39 AM UTC. 2023-01-25T08:55:56.187 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-01-25T08:55:56.188 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:56.188 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2023-01-25T08:55:56.188 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:56.188 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-01-25T08:55:56.189 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T08:55:56.189 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:56.189 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-01-25T08:55:56.189 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-01-25T08:55:56.189 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2023-01-25T08:55:56.190 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:56.190 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 180 k 2023-01-25T08:55:56.190 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 568 k 2023-01-25T08:55:56.190 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-01-25T08:55:56.652 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 388 kB/s | 180 kB 00:00 2023-01-25T08:55:56.653 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-01-25T08:55:56.653 INFO:teuthology.orchestra.run.smithi008.stdout:Total 387 kB/s | 180 kB 00:00 2023-01-25T08:55:56.653 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-01-25T08:55:56.666 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-01-25T08:55:56.666 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-01-25T08:55:56.721 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-01-25T08:55:56.725 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-01-25T08:55:56.850 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-01-25T08:55:57.014 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:57.331 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:57.817 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T08:55:57.817 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:57.817 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-01-25T08:55:57.817 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T08:55:57.818 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:55:57.818 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-01-25T08:55:58.007 DEBUG:teuthology.parallel:result is None 2023-01-25T08:55:58.008 DEBUG:teuthology.orchestra.run.smithi006:> sudo lsb_release -is 2023-01-25T08:55:58.054 INFO:teuthology.orchestra.run.smithi006.stdout:CentOSStream 2023-01-25T08:55:58.055 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:55:58.056 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T08:55:58.401 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T08:55:58.429 INFO:teuthology.orchestra.run.smithi006.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T08:55:58.430 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T08:55:58.430 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T08:55:58.431 DEBUG:teuthology.orchestra.run.smithi008:> sudo lsb_release -is 2023-01-25T08:55:58.470 INFO:teuthology.orchestra.run.smithi008.stdout:CentOSStream 2023-01-25T08:55:58.472 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T08:55:58.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T08:55:58.803 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T08:55:58.828 INFO:teuthology.orchestra.run.smithi008.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T08:55:58.828 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T08:55:58.829 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T08:55:58.830 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T08:55:58.830 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:55:58.831 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T08:55:58.866 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:55:58.866 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T08:55:58.896 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T08:55:58.897 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:55:58.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T08:55:58.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T08:55:59.009 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:55:59.009 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T08:55:59.037 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T08:55:59.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T08:55:59.106 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:55:59.107 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T08:55:59.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T08:55:59.207 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:55:59.208 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T08:55:59.236 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T08:55:59.304 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T08:55:59.370 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T08:55:59.371 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /var/log/ceph 2023-01-25T08:55:59.374 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /var/log/ceph 2023-01-25T08:55:59.402 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T08:55:59.402 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T08:55:59.445 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T08:55:59.473 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T08:55:59.473 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T08:55:59.516 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T08:55:59.555 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T08:55:59.556 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2023-01-25T08:55:59.556 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137121', 'branch': 'wip-yuri6-testing-2023-01-24-0746', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache-pool-snaps-readproxy}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7137121', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2023-01-24-0746'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2023-01-24-0746', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2023-01-24-0746', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'targets': {'smithi006.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCefLXCGwe7hJJcQWjUvgCvKeQiI/mno2g4dps6sLEuBk4FgmbQB0Hd/fw+tPJg0LnxKeInVURif1yp2pe5vDOUB/nkwcCiN0POfXDGAHjMlDAil9Vx7Iui5UlX7Kc2qHMEGEQ6s3IEXLVXBk7mRItZ4pf/y8gixZTQeU3onF7qibPrRkLUWPmg3QIpgmA282RMdWZaQC4CLRyinDhH9+DfEpx4DEor7k5a/jmdVPYwsRJ6EI8WbP9unwJVKtW5Y0lXyyKf9NAe0zUPu54J6/TcqaDQANMMkROnRc4hhD24BEyFSpafeemPju/0tQVZ0p9yyPI2VvjFcFCE/uDycJRHM2CX4tLMWBVF0YUUTzxN28bRpXR17nK9vSVjK/3CqTNeV92iE3pJTR1yLmqn2i9Fa3rEwgF8Y5KWfDZoQ5nJB39lx8B8HhZ7Fu7KOO5iElT68FqLSACeNZH+VmBpwzGWavBPt+lmoe6xfr3lpfSLwfJwy9/Qbg6Xo2JzDeeJfwc=', 'smithi008.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbBVW0ZM4KXHL0z52DF2cenLOEd3+t35UP8aqMMuL5YV2MGE1Q+WNeVWi6V7c9WXLg/meRERB3mS21t9dCdn7ZbrKGQCiN9oHFVRqlKdCijfHNkV5r1d/Y1eDFM7QTKoTMRNIVrrjCsW/48iNLOStilZxhUCsTpapeBwcYf7XnWGHY+aawuWeEaD7SmOuiRJRT8VuOBlDDXxVR2NRHf86P/xk6vFCP/VfmzkRCud8COElz0rh2mubJ0qv1Zx8qghzJnP1/AoMK+ggbORL93MDIfSib+WRV718ini5EHM+NBR6yIVfqey2zHE6VQrPWnA4ZZpPYRCA7iUHRXejIoKRh34PC2U/q6+q0YBo340u/T5hwd3fOsGYLt6vVNQV7QJLULvdHWWzlna6keh1VaeSDchfK2IxlkVOaiC5W3qHKXH9br2+uUpshjcU+k65hoLo2DlcSPlPh4DVShoVLuatgNYy/9NiEw0ZpJWjeVRfO+DIMcBc15yhRJCegZiuP+Qc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache readproxy', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:27:30', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T08:55:59.557 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T08:55:59.586 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T08:55:59.604 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T08:55:59.646 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T08:55:59.674 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:55:59.674 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T08:55:59.733 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T08:55:59.734 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2023-01-25T08:55:59.792 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T08:55:59.793 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:55:59.793 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 372830 Links: 1 2023-01-25T08:55:59.793 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:55:59.793 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:55:59.794 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-01-25 08:55:30.865334993 +0000 2023-01-25T08:55:59.794 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-01-25 08:51:44.025812738 +0000 2023-01-25T08:55:59.794 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-01-25 08:51:44.025812738 +0000 2023-01-25T08:55:59.794 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-01-25T08:55:59.795 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T08:55:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-01-25T08:55:59.860 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-01-25T08:55:59.861 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000320019 s, 1.6 MB/s 2023-01-25T08:55:59.862 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T08:55:59.921 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2023-01-25T08:55:59.980 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T08:55:59.981 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:55:59.981 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 373892 Links: 1 2023-01-25T08:55:59.981 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:55:59.981 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:55:59.982 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-01-25 08:55:30.865334993 +0000 2023-01-25T08:55:59.982 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-01-25 08:51:44.522807758 +0000 2023-01-25T08:55:59.982 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-01-25 08:51:44.522807758 +0000 2023-01-25T08:55:59.982 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-01-25T08:55:59.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T08:56:00.048 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-01-25T08:56:00.048 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-01-25T08:56:00.048 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000209094 s, 2.4 MB/s 2023-01-25T08:56:00.049 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T08:56:00.109 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2023-01-25T08:56:00.168 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T08:56:00.168 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:00.169 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 376035 Links: 1 2023-01-25T08:56:00.169 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:00.169 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:00.169 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-01-25 08:55:30.865334993 +0000 2023-01-25T08:56:00.170 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-01-25 08:51:45.038802587 +0000 2023-01-25T08:56:00.170 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-01-25 08:51:45.038802587 +0000 2023-01-25T08:56:00.170 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-01-25T08:56:00.170 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T08:56:00.236 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-01-25T08:56:00.237 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-01-25T08:56:00.237 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000214242 s, 2.4 MB/s 2023-01-25T08:56:00.239 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T08:56:00.299 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2023-01-25T08:56:00.358 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T08:56:00.358 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:00.358 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 375102 Links: 1 2023-01-25T08:56:00.358 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:00.359 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:00.359 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-01-25 08:55:30.865334993 +0000 2023-01-25T08:56:00.359 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-01-25 08:51:45.544797517 +0000 2023-01-25T08:56:00.359 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-01-25 08:51:45.544797517 +0000 2023-01-25T08:56:00.359 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-01-25T08:56:00.360 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T08:56:00.425 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-01-25T08:56:00.425 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-01-25T08:56:00.426 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000179383 s, 2.9 MB/s 2023-01-25T08:56:00.427 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T08:56:00.486 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:00.487 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:00.487 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T08:56:00.507 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T08:56:00.508 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2023-01-25T08:56:00.567 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T08:56:00.568 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:00.568 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 364375 Links: 1 2023-01-25T08:56:00.569 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:00.569 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:00.569 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-01-25 08:55:53.197156064 +0000 2023-01-25T08:56:00.570 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-01-25 08:51:44.019123552 +0000 2023-01-25T08:56:00.570 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-01-25 08:51:44.019123552 +0000 2023-01-25T08:56:00.570 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-01-25T08:56:00.571 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T08:56:00.636 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-01-25T08:56:00.637 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-01-25T08:56:00.637 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000107763 s, 4.8 MB/s 2023-01-25T08:56:00.638 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T08:56:00.698 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2023-01-25T08:56:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T08:56:00.759 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:00.760 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 374890 Links: 1 2023-01-25T08:56:00.760 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:00.760 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:00.761 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-01-25 08:55:53.198156053 +0000 2023-01-25T08:56:00.761 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-01-25 08:51:44.526117906 +0000 2023-01-25T08:56:00.761 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-01-25 08:51:44.526117906 +0000 2023-01-25T08:56:00.762 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-01-25T08:56:00.762 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T08:56:00.826 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-01-25T08:56:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-01-25T08:56:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000105407 s, 4.9 MB/s 2023-01-25T08:56:00.828 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T08:56:00.888 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2023-01-25T08:56:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T08:56:00.948 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:00.948 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 371228 Links: 1 2023-01-25T08:56:00.948 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:00.949 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:00.949 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-01-25 08:55:53.198156053 +0000 2023-01-25T08:56:00.949 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-01-25 08:51:45.030112292 +0000 2023-01-25T08:56:00.950 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-01-25 08:51:45.030112292 +0000 2023-01-25T08:56:00.950 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-01-25T08:56:00.950 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T08:56:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-01-25T08:56:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-01-25T08:56:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000105165 s, 4.9 MB/s 2023-01-25T08:56:01.016 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T08:56:01.075 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2023-01-25T08:56:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T08:56:01.133 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T08:56:01.134 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 371314 Links: 1 2023-01-25T08:56:01.134 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T08:56:01.134 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T08:56:01.134 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-01-25 08:55:53.198156053 +0000 2023-01-25T08:56:01.134 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-01-25 08:51:45.541106601 +0000 2023-01-25T08:56:01.135 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-01-25 08:51:45.541106601 +0000 2023-01-25T08:56:01.135 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2023-01-25T08:56:01.135 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T08:56:01.200 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-01-25T08:56:01.200 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-01-25T08:56:01.200 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 9.8068e-05 s, 5.2 MB/s 2023-01-25T08:56:01.201 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T08:56:01.261 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:01.261 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T08:56:01.261 INFO:tasks.ceph:Generating config... 2023-01-25T08:56:01.263 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T08:56:01.263 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-25T08:56:01.263 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2023-01-25T08:56:01.263 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-25T08:56:01.264 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-25T08:56:01.264 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T08:56:01.264 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-25T08:56:01.264 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-25T08:56:01.265 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T08:56:01.265 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T08:56:01.265 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-25T08:56:01.266 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-25T08:56:01.266 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-25T08:56:01.266 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-25T08:56:01.266 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T08:56:01.267 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2023-01-25T08:56:01.267 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T08:56:01.267 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T08:56:01.267 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-25T08:56:01.268 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2023-01-25T08:56:01.268 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-01-25T08:56:01.268 INFO:tasks.ceph:[osd] osd max backfills = 3 2023-01-25T08:56:01.269 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T08:56:01.269 INFO:tasks.ceph:[osd] osd max scrubs = 3 2023-01-25T08:56:01.269 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T08:56:01.269 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-25T08:56:01.270 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T08:56:01.270 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T08:56:01.270 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-25T08:56:01.270 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-25T08:56:01.271 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T08:56:01.271 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2023-01-25T08:56:01.271 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T08:56:01.271 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T08:56:01.272 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T08:56:01.272 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-25T08:56:01.272 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-01-25T08:56:01.273 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-01-25T08:56:01.273 INFO:tasks.ceph:[global] ms type = async 2023-01-25T08:56:01.273 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2023-01-25T08:56:01.273 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2023-01-25T08:56:01.274 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T08:56:01.274 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-01-25T08:56:01.274 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T08:56:01.274 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T08:56:01.275 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T08:56:01.275 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T08:56:01.275 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T08:56:01.276 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-25T08:56:01.276 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-25T08:56:01.276 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-25T08:56:01.276 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-25T08:56:01.277 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T08:56:01.277 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-25T08:56:01.277 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T08:56:01.278 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T08:56:01.336 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T08:56:01.339 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T08:56:01.385 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T08:56:01.413 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.6:6789]'), ('mon.c', '[v1:172.21.15.6:6790]'), ('mon.b', '[v1:172.21.15.8:6789]')] 2023-01-25T08:56:01.414 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.6:6789],[v1:172.21.15.6:6790],[v1:172.21.15.8:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T08:56:01.415 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:01.415 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T08:56:01.472 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.6:6789]' --addv c '[v1:172.21.15.6:6790]' --addv b '[v1:172.21.15.8:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:01.561 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:01.561 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: generated fsid a4dc1c97-bb3d-4eba-b23f-bf25be3327d8 2023-01-25T08:56:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = pacific 2023-01-25T08:56:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 0 2023-01-25T08:56:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:fsid a4dc1c97-bb3d-4eba-b23f-bf25be3327d8 2023-01-25T08:56:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:last_changed 2023-01-25T08:56:01.561302+0000 2023-01-25T08:56:01.562 INFO:teuthology.orchestra.run.smithi006.stdout:created 2023-01-25T08:56:01.561302+0000 2023-01-25T08:56:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:min_mon_release 16 (pacific) 2023-01-25T08:56:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:election_strategy: 1 2023-01-25T08:56:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:0: v1:172.21.15.6:6789/0 mon.a 2023-01-25T08:56:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:1: v1:172.21.15.8:6789/0 mon.b 2023-01-25T08:56:01.563 INFO:teuthology.orchestra.run.smithi006.stdout:2: v1:172.21.15.6:6790/0 mon.c 2023-01-25T08:56:01.564 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T08:56:01.565 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T08:56:01.583 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a4dc1c97-bb3d-4eba-b23f-bf25be3327d8... 2023-01-25T08:56:01.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T08:56:01.626 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T08:56:01.670 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2023-01-25T08:56:01.670 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2023-01-25T08:56:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T08:56:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2023-01-25T08:56:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.671 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2023-01-25T08:56:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2023-01-25T08:56:01.672 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2023-01-25T08:56:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2023-01-25T08:56:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.673 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2023-01-25T08:56:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2023-01-25T08:56:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.674 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T08:56:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2023-01-25T08:56:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.675 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T08:56:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T08:56:01.676 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2023-01-25T08:56:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.677 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2023-01-25T08:56:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2023-01-25T08:56:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2023-01-25T08:56:01.678 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2023-01-25T08:56:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2023-01-25T08:56:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2023-01-25T08:56:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2023-01-25T08:56:01.679 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T08:56:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T08:56:01.680 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2023-01-25T08:56:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T08:56:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2023-01-25T08:56:01.681 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2023-01-25T08:56:01.682 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "[v1:172.21.15.6:6789],[v1:172.21.15.6:6790],[v1:172.21.15.8:6789]" 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: mon client directed command retry = 5 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: mon election default strategy = 1 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: ms bind msgr2 = False 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: ms inject socket failures = 5000 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: ms type = async 2023-01-25T08:56:01.683 INFO:teuthology.orchestra.run.smithi006.stdout: osd_async_recovery_min_cost = 1 2023-01-25T08:56:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-25T08:56:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_min_size = 2 2023-01-25T08:56:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_size = 2 2023-01-25T08:56:01.684 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = a4dc1c97-bb3d-4eba-b23f-bf25be3327d8 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2023-01-25T08:56:01.685 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 120 2023-01-25T08:56:01.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.686 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2023-01-25T08:56:01.686 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2023-01-25T08:56:01.686 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.686 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2023-01-25T08:56:01.687 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2023-01-25T08:56:01.687 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2023-01-25T08:56:01.687 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.687 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2023-01-25T08:56:01.687 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2023-01-25T08:56:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2023-01-25T08:56:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2023-01-25T08:56:01.688 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2023-01-25T08:56:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2023-01-25T08:56:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2023-01-25T08:56:01.689 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2023-01-25T08:56:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2023-01-25T08:56:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: debug monc = 20 2023-01-25T08:56:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2023-01-25T08:56:01.690 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore compression algorithm = zstd 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore compression mode = aggressive 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore zero block detection = True 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2023-01-25T08:56:01.691 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-01-25T08:56:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2023-01-25T08:56:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2023-01-25T08:56:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T08:56:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2023-01-25T08:56:01.692 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd blocked scrub grace period = 3600 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug reject backfill probability = 0.3 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify cached snaps = True 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify missing on start = True 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2023-01-25T08:56:01.693 INFO:teuthology.orchestra.run.smithi006.stdout: osd heartbeat use min delay socket = True 2023-01-25T08:56:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: osd inject bad map crc probability = 0.1 2023-01-25T08:56:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: osd max backfills = 3 2023-01-25T08:56:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: osd max markdown count = 1000 2023-01-25T08:56:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: osd max scrubs = 3 2023-01-25T08:56:01.694 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock override recovery settings = True 2023-01-25T08:56:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2023-01-25T08:56:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: osd op queue = debug_random 2023-01-25T08:56:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: osd op queue cut off = debug_random 2023-01-25T08:56:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub min interval = 60 2023-01-25T08:56:01.695 INFO:teuthology.orchestra.run.smithi006.stdout: osd shutdown pgref assert = True 2023-01-25T08:56:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: osd snap trim sleep = 2 2023-01-25T08:56:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: osd_max_pg_log_entries = 3000 2023-01-25T08:56:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: osd_min_pg_log_entries = 3000 2023-01-25T08:56:01.696 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.696 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2023-01-25T08:56:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-01-25T08:56:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2023-01-25T08:56:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-01-25T08:56:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-01-25T08:56:01.697 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-01-25T08:56:01.698 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-01-25T08:56:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2023-01-25T08:56:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-01-25T08:56:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2023-01-25T08:56:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2023-01-25T08:56:01.699 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-01-25T08:56:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2023-01-25T08:56:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2023-01-25T08:56:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-01-25T08:56:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.700 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T08:56:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T08:56:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T08:56:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T08:56:01.701 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T08:56:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T08:56:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2023-01-25T08:56:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2023-01-25T08:56:01.702 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon min osdmap epochs = 50 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon osdmap full prune interval = 2 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon osdmap full prune min = 15 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon osdmap full prune txsize = 2 2023-01-25T08:56:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: mon scrub interval = 300 2023-01-25T08:56:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: paxos service trim min = 10 2023-01-25T08:56:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:01.704 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2023-01-25T08:56:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2023-01-25T08:56:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2023-01-25T08:56:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2023-01-25T08:56:01.705 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T08:56:01.706 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T08:56:01.706 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2023-01-25T08:56:01.706 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2023-01-25T08:56:01.707 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2023-01-25T08:56:01.709 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2023-01-25T08:56:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: chdir = "" 2023-01-25T08:56:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T08:56:01.710 INFO:teuthology.orchestra.run.smithi008.stdout: auth supported = cephx 2023-01-25T08:56:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: filestore xattr use omap = true 2023-01-25T08:56:01.711 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: mon clock drift allowed = 1.000 2023-01-25T08:56:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.712 INFO:teuthology.orchestra.run.smithi008.stdout: osd crush chooseleaf type = 0 2023-01-25T08:56:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: auth debug = true 2023-01-25T08:56:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on old message = true 2023-01-25T08:56:01.713 INFO:teuthology.orchestra.run.smithi008.stdout: ms die on bug = true 2023-01-25T08:56:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T08:56:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: mon pg warn max object skew = 0 2023-01-25T08:56:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.714 INFO:teuthology.orchestra.run.smithi008.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T08:56:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T08:56:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default size = 2 2023-01-25T08:56:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.715 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow primary affinity = true 2023-01-25T08:56:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd allow pg remap = true 2023-01-25T08:56:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on legacy crush tunables = false 2023-01-25T08:56:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on crush straw calc version zero = false 2023-01-25T08:56:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on no sortbitwise = false 2023-01-25T08:56:01.716 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on osd down out interval zero = false 2023-01-25T08:56:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: mon warn on too few osds = false 2023-01-25T08:56:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T08:56:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T08:56:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: mon_allow_pool_size_one = true 2023-01-25T08:56:01.717 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T08:56:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: osd default data pool replay window = 5 2023-01-25T08:56:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.718 INFO:teuthology.orchestra.run.smithi008.stdout: mon allow pool delete = true 2023-01-25T08:56:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: mon cluster log file level = debug 2023-01-25T08:56:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: debug asserts on shutdown = true 2023-01-25T08:56:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: mon health detail to clog = false 2023-01-25T08:56:01.719 INFO:teuthology.orchestra.run.smithi008.stdout: mon host = "[v1:172.21.15.6:6789],[v1:172.21.15.6:6790],[v1:172.21.15.8:6789]" 2023-01-25T08:56:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: mon client directed command retry = 5 2023-01-25T08:56:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: mon election default strategy = 1 2023-01-25T08:56:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: ms bind msgr2 = False 2023-01-25T08:56:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: ms inject socket failures = 5000 2023-01-25T08:56:01.720 INFO:teuthology.orchestra.run.smithi008.stdout: ms type = async 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: osd_async_recovery_min_cost = 1 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: osd_object_clean_region_max_num_intervals = 1000 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: osd_pool_default_min_size = 2 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: osd_pool_default_size = 2 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = a4dc1c97-bb3d-4eba-b23f-bf25be3327d8 2023-01-25T08:56:01.721 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.722 INFO:teuthology.orchestra.run.smithi008.stdout:[osd] 2023-01-25T08:56:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: osd journal size = 100 2023-01-25T08:56:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub load threshold = 5.0 2023-01-25T08:56:01.722 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub max interval = 120 2023-01-25T08:56:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: osd recover clone overlap = true 2023-01-25T08:56:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: osd recovery max chunk = 1048576 2023-01-25T08:56:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.723 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug shutdown = true 2023-01-25T08:56:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug op order = true 2023-01-25T08:56:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug verify stray on activate = true 2023-01-25T08:56:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: osd open classes on start = true 2023-01-25T08:56:01.724 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug pg log writeout = true 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: osd deep scrub update digest min age = 30 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: osd map max advance = 10 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.725 INFO:teuthology.orchestra.run.smithi008.stdout: journal zero on create = true 2023-01-25T08:56:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: filestore ondisk finisher threads = 3 2023-01-25T08:56:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: filestore apply finisher threads = 3 2023-01-25T08:56:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.726 INFO:teuthology.orchestra.run.smithi008.stdout: bdev debug aio = true 2023-01-25T08:56:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug misdirected ops = true 2023-01-25T08:56:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: debug monc = 20 2023-01-25T08:56:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore block size = 96636764160 2023-01-25T08:56:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore compression algorithm = zstd 2023-01-25T08:56:01.727 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore compression mode = aggressive 2023-01-25T08:56:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore fsck on mount = True 2023-01-25T08:56:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: bluestore zero block detection = True 2023-01-25T08:56:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluefs = 20 2023-01-25T08:56:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: debug bluestore = 20 2023-01-25T08:56:01.728 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: debug osd = 20 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: debug rocksdb = 10 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd full ratio = 0.9 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd nearfull ratio = 0.8 2023-01-25T08:56:01.729 INFO:teuthology.orchestra.run.smithi008.stdout: osd blocked scrub grace period = 3600 2023-01-25T08:56:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug reject backfill probability = 0.3 2023-01-25T08:56:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug verify cached snaps = True 2023-01-25T08:56:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: osd debug verify missing on start = True 2023-01-25T08:56:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: osd failsafe full ratio = 0.95 2023-01-25T08:56:01.730 INFO:teuthology.orchestra.run.smithi008.stdout: osd heartbeat use min delay socket = True 2023-01-25T08:56:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: osd inject bad map crc probability = 0.1 2023-01-25T08:56:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: osd max backfills = 3 2023-01-25T08:56:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: osd max markdown count = 1000 2023-01-25T08:56:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: osd max scrubs = 3 2023-01-25T08:56:01.731 INFO:teuthology.orchestra.run.smithi008.stdout: osd mclock override recovery settings = True 2023-01-25T08:56:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: osd objectstore = bluestore 2023-01-25T08:56:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: osd op queue = debug_random 2023-01-25T08:56:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: osd op queue cut off = debug_random 2023-01-25T08:56:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: osd scrub min interval = 60 2023-01-25T08:56:01.732 INFO:teuthology.orchestra.run.smithi008.stdout: osd shutdown pgref assert = True 2023-01-25T08:56:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: osd snap trim sleep = 2 2023-01-25T08:56:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: osd_max_pg_log_entries = 3000 2023-01-25T08:56:01.733 INFO:teuthology.orchestra.run.smithi008.stdout: osd_min_pg_log_entries = 3000 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr] 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: debug mgr = 20 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-01-25T08:56:01.734 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-01-25T08:56:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-01-25T08:56:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-01-25T08:56:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: mgr/telemetry/nag = false 2023-01-25T08:56:01.735 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.735 INFO:teuthology.orchestra.run.smithi008.stdout:[mon] 2023-01-25T08:56:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: debug ms = 1 2023-01-25T08:56:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: debug mon = 20 2023-01-25T08:56:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: debug paxos = 20 2023-01-25T08:56:01.736 INFO:teuthology.orchestra.run.smithi008.stdout: debug auth = 20 2023-01-25T08:56:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: mon data avail warn = 5 2023-01-25T08:56:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: mon mgr mkfs grace = 240 2023-01-25T08:56:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min pgs per osd = 4 2023-01-25T08:56:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd reporter subtree level = osd 2023-01-25T08:56:01.737 INFO:teuthology.orchestra.run.smithi008.stdout: mon osd prime pg temp = true 2023-01-25T08:56:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: mon reweight min bytes per osd = 10 2023-01-25T08:56:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T08:56:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T08:56:01.738 INFO:teuthology.orchestra.run.smithi008.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T08:56:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T08:56:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T08:56:01.739 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T08:56:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: # 1m isn't quite enough 2023-01-25T08:56:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: mon_down_mkfs_grace = 2m 2023-01-25T08:56:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.740 INFO:teuthology.orchestra.run.smithi008.stdout: mon_warn_on_filestore_osds = false 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: mon min osdmap epochs = 50 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: mon osdmap full prune interval = 2 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: mon osdmap full prune min = 15 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: mon osdmap full prune txsize = 2 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: mon scrub interval = 300 2023-01-25T08:56:01.741 INFO:teuthology.orchestra.run.smithi008.stdout: paxos service trim min = 10 2023-01-25T08:56:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-01-25T08:56:01.742 INFO:teuthology.orchestra.run.smithi008.stdout:[client] 2023-01-25T08:56:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: rgw cache enabled = true 2023-01-25T08:56:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable ops log = true 2023-01-25T08:56:01.742 INFO:teuthology.orchestra.run.smithi008.stdout: rgw enable usage log = true 2023-01-25T08:56:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T08:56:01.743 INFO:teuthology.orchestra.run.smithi008.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T08:56:01.743 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.a] 2023-01-25T08:56:01.743 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.c] 2023-01-25T08:56:01.743 INFO:teuthology.orchestra.run.smithi008.stdout:[mon.b] 2023-01-25T08:56:01.744 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T08:56:01.744 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T08:56:01.803 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T08:56:01.803 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:01.804 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T08:56:01.822 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:01.822 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T08:56:01.880 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:01.881 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:01.881 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T08:56:01.882 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T08:56:01.958 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:01.959 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:02.017 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:56:02.017 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:02.017 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T08:56:02.018 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T08:56:02.057 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:02.058 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:02.117 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T08:56:02.118 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T08:56:02.118 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T08:56:02.174 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T08:56:02.176 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T08:56:02.240 INFO:teuthology.orchestra.run.smithi008.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T08:56:02.242 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T08:56:02.243 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T08:56:02.243 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T08:56:02.289 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T08:56:02.302 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T08:56:02.348 INFO:teuthology.orchestra.run.smithi008.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T08:56:02.361 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T08:56:02.361 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi008.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T08:56:02.361 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T08:56:02.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:02.390 INFO:tasks.ceph:role: osd.0 2023-01-25T08:56:02.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:02.391 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T08:56:02.465 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:02.466 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:02.466 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:02.467 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:02.467 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:02.467 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:02.467 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:02.899 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-01-25T08:56:02.901 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-01-25T08:56:02.901 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T08:56:03.059 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T08:56:03.097 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T08:56:03.166 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:03.166 INFO:tasks.ceph:role: osd.1 2023-01-25T08:56:03.166 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:03.167 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T08:56:03.242 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:03.243 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:03.244 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:03.244 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:03.244 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:03.660 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-01-25T08:56:03.662 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-01-25T08:56:03.663 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T08:56:03.769 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T08:56:03.803 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T08:56:03.872 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:03.873 INFO:tasks.ceph:role: osd.2 2023-01-25T08:56:03.873 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:03.873 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T08:56:03.948 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:03.949 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:03.949 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:03.949 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:03.950 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:03.950 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:03.950 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:03.950 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:03.950 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:03.951 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:04.393 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-01-25T08:56:04.395 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-01-25T08:56:04.396 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T08:56:04.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T08:56:04.577 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T08:56:04.646 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:04.647 INFO:tasks.ceph:role: osd.3 2023-01-25T08:56:04.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:04.647 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T08:56:04.723 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:04.723 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:04.723 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:04.723 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:04.724 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:04.724 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:04.724 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:04.724 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:04.724 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:04.725 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:05.158 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-01-25T08:56:05.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-01-25T08:56:05.160 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T08:56:05.350 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T08:56:05.384 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:05.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.592+0000 7fd9722db580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T08:56:05.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.592+0000 7fd9722db580 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T08:56:05.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.592+0000 7fd9722db580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T08:56:05.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.592+0000 7fd9722db580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T08:56:05.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.592+0000 7fd9722db580 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T08:56:05.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:05.593+0000 7fd9722db580 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T08:56:06.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:06.119+0000 7fd9722db580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:06.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:06.881+0000 7fd9722db580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:07.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:07.646+0000 7fd9722db580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:08.442 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T08:56:08.477 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:08.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.570+0000 7ff50c170580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T08:56:08.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.571+0000 7ff50c170580 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T08:56:08.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.571+0000 7ff50c170580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T08:56:08.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.571+0000 7ff50c170580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T08:56:08.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.571+0000 7ff50c170580 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T08:56:08.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:08.571+0000 7ff50c170580 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T08:56:09.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:09.096+0000 7ff50c170580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:09.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:09.857+0000 7ff50c170580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:10.627+0000 7ff50c170580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:11.423 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T08:56:11.457 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:11.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.549+0000 7fe5adc5c580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T08:56:11.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.550+0000 7fe5adc5c580 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T08:56:11.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.550+0000 7fe5adc5c580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T08:56:11.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.550+0000 7fe5adc5c580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T08:56:11.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.550+0000 7fe5adc5c580 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T08:56:11.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:11.550+0000 7fe5adc5c580 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T08:56:12.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:12.077+0000 7fe5adc5c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:12.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:12.843+0000 7fe5adc5c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:13.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:13.606+0000 7fe5adc5c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:14.403 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T08:56:14.437 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:14.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.528+0000 7f567821f580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T08:56:14.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.528+0000 7f567821f580 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T08:56:14.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.529+0000 7f567821f580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T08:56:14.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.529+0000 7f567821f580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T08:56:14.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.529+0000 7f567821f580 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T08:56:14.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:14.529+0000 7f567821f580 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T08:56:15.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:15.055+0000 7f567821f580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:15.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:15.817+0000 7f567821f580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:16.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-01-25T08:56:16.581+0000 7f567821f580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:17.384 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T08:56:17.418 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T08:56:17.453 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:17.453 INFO:tasks.ceph:role: osd.4 2023-01-25T08:56:17.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:56:17.454 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T08:56:17.525 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:17.525 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:17.526 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:17.526 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:17.526 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:17.526 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:17.526 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:17.527 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:17.527 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:17.527 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:17.548 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-01-25T08:56:17.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-01-25T08:56:17.550 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T08:56:17.634 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T08:56:17.708 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T08:56:17.778 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:17.778 INFO:tasks.ceph:role: osd.5 2023-01-25T08:56:17.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:56:17.779 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T08:56:17.845 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:17.846 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:17.846 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:17.846 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:17.846 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:17.846 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:17.847 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:17.847 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:17.847 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:17.847 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:17.868 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-01-25T08:56:17.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-01-25T08:56:17.870 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T08:56:17.951 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T08:56:18.023 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T08:56:18.092 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:18.092 INFO:tasks.ceph:role: osd.6 2023-01-25T08:56:18.093 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:56:18.093 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T08:56:18.160 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:18.161 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:18.161 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:18.161 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:18.161 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:18.161 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:18.162 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:18.162 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:18.162 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:18.162 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:18.183 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-01-25T08:56:18.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-01-25T08:56:18.185 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T08:56:18.266 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T08:56:18.338 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T08:56:18.406 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T08:56:18.406 INFO:tasks.ceph:role: osd.7 2023-01-25T08:56:18.407 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi008.front.sepia.ceph.com 2023-01-25T08:56:18.407 DEBUG:teuthology.orchestra.run.smithi008:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T08:56:18.474 INFO:teuthology.orchestra.run.smithi008.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T08:56:18.474 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T08:56:18.475 INFO:teuthology.orchestra.run.smithi008.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T08:56:18.475 INFO:teuthology.orchestra.run.smithi008.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T08:56:18.475 INFO:teuthology.orchestra.run.smithi008.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T08:56:18.475 INFO:teuthology.orchestra.run.smithi008.stdout: = sunit=0 swidth=0 blks 2023-01-25T08:56:18.475 INFO:teuthology.orchestra.run.smithi008.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T08:56:18.476 INFO:teuthology.orchestra.run.smithi008.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T08:56:18.476 INFO:teuthology.orchestra.run.smithi008.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T08:56:18.476 INFO:teuthology.orchestra.run.smithi008.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T08:56:18.497 INFO:teuthology.orchestra.run.smithi008.stdout:Discarding blocks...Done. 2023-01-25T08:56:18.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi008.front.sepia.ceph.com -o noatime 2023-01-25T08:56:18.499 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T08:56:18.580 DEBUG:teuthology.orchestra.run.smithi008:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T08:56:18.651 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:18.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T08:56:18.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T08:56:18.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T08:56:18.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T08:56:18.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T08:56:18.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:18.743+0000 7fb70f7a8580 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T08:56:19.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:19.267+0000 7fb70f7a8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:20.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:20.026+0000 7fb70f7a8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:20.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:20.789+0000 7fb70f7a8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:21.585 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T08:56:21.619 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:21.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.711+0000 7f55d36f8580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T08:56:21.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.711+0000 7f55d36f8580 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T08:56:21.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.711+0000 7f55d36f8580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T08:56:21.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.711+0000 7f55d36f8580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T08:56:21.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.711+0000 7f55d36f8580 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T08:56:21.715 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:21.712+0000 7f55d36f8580 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T08:56:22.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:22.235+0000 7f55d36f8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:22.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:22.996+0000 7f55d36f8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:23.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:23.759+0000 7f55d36f8580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:24.553 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T08:56:24.588 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.681+0000 7f3ae2930580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T08:56:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.681+0000 7f3ae2930580 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T08:56:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.681+0000 7f3ae2930580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T08:56:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.681+0000 7f3ae2930580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T08:56:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.681+0000 7f3ae2930580 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T08:56:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:24.682+0000 7f3ae2930580 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T08:56:25.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:25.210+0000 7f3ae2930580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:25.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:25.969+0000 7f3ae2930580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:26.736+0000 7f3ae2930580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:27.541 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T08:56:27.575 DEBUG:teuthology.orchestra.run.smithi008:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T08:56:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T08:56:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T08:56:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T08:56:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T08:56:27.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:27.666+0000 7fdd4cb6c580 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T08:56:28.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:28.191+0000 7fdd4cb6c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:28.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:28.951+0000 7fdd4cb6c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:29.717 INFO:teuthology.orchestra.run.smithi008.stderr:2023-01-25T08:56:29.716+0000 7fdd4cb6c580 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T08:56:30.510 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T08:56:30.544 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T08:56:30.544 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:30.544 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T08:56:30.581 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:30.582 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T08:56:30.655 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:30.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T08:56:30.727 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:30.727 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T08:56:30.798 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:30.799 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T08:56:30.869 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:30.870 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T08:56:30.901 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:30.902 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T08:56:30.973 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:30.973 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T08:56:31.044 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:31.045 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T08:56:31.115 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:31.115 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T08:56:31.185 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:31.185 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T08:56:31.205 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:31.206 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T08:56:31.244 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T08:56:31.245 DEBUG:teuthology.orchestra.run.smithi006:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T08:56:31.248 DEBUG:teuthology.orchestra.run.smithi008:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T08:56:31.288 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.y] 2023-01-25T08:56:31.288 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCi7tBjJUFiChAAq8XWqkGCTy+8I/kvkxoojA== 2023-01-25T08:56:31.288 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.0] 2023-01-25T08:56:31.289 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCl7tBjBRxmIxAAdPN7galD2J7miPOpUAMuew== 2023-01-25T08:56:31.289 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.1] 2023-01-25T08:56:31.289 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCo7tBjyaYbIhAAJoOIbFI1nH2573G1/PvXWA== 2023-01-25T08:56:31.289 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.2] 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCr7tBjmpvbIBAAAM10Q0ulZOanMnr5Q9sZ4A== 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.3] 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCu7tBj8++YHxAARfNhrWbCurX5Jfx/1xqdYg== 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.x] 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCi7tBjRVRPDhAA6qU5utWQdjJd3CNKk/1haQ== 2023-01-25T08:56:31.290 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.4] 2023-01-25T08:56:31.291 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCy7tBjdjVlLBAA+au4l6gB0L2E6iIt5a2UhQ== 2023-01-25T08:56:31.291 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.5] 2023-01-25T08:56:31.291 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQC17tBjaziGKhAAZJcRGDkNyvQnIMUs9ZVQ5A== 2023-01-25T08:56:31.291 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.6] 2023-01-25T08:56:31.291 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQC47tBjo0e9KBAAp/huYVA0Dq8PUGEMfBRNWg== 2023-01-25T08:56:31.292 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.7] 2023-01-25T08:56:31.292 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQC77tBjbrvRJxAAWYRgsbZ8CbXMxfw1BLmIrQ== 2023-01-25T08:56:31.292 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2023-01-25T08:56:31.292 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCi7tBjpOw8ERAAsJrwUmdFtrJ7Rdu7D5tpew== 2023-01-25T08:56:31.293 INFO:teuthology.orchestra.run.smithi006.stdout:[client.1] 2023-01-25T08:56:31.293 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQCi7tBjIMvAFBAAgkqW5m08tMaFMzKnuHlIIQ== 2023-01-25T08:56:31.313 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr.y] 2023-01-25T08:56:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCi7tBjJUFiChAAq8XWqkGCTy+8I/kvkxoojA== 2023-01-25T08:56:31.313 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.0] 2023-01-25T08:56:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCl7tBjBRxmIxAAdPN7galD2J7miPOpUAMuew== 2023-01-25T08:56:31.314 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.1] 2023-01-25T08:56:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCo7tBjyaYbIhAAJoOIbFI1nH2573G1/PvXWA== 2023-01-25T08:56:31.314 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.2] 2023-01-25T08:56:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCr7tBjmpvbIBAAAM10Q0ulZOanMnr5Q9sZ4A== 2023-01-25T08:56:31.315 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.3] 2023-01-25T08:56:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCu7tBj8++YHxAARfNhrWbCurX5Jfx/1xqdYg== 2023-01-25T08:56:31.315 INFO:teuthology.orchestra.run.smithi008.stdout:[mgr.x] 2023-01-25T08:56:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCi7tBjRVRPDhAA6qU5utWQdjJd3CNKk/1haQ== 2023-01-25T08:56:31.315 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.4] 2023-01-25T08:56:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCy7tBjdjVlLBAA+au4l6gB0L2E6iIt5a2UhQ== 2023-01-25T08:56:31.316 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.5] 2023-01-25T08:56:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQC17tBjaziGKhAAZJcRGDkNyvQnIMUs9ZVQ5A== 2023-01-25T08:56:31.316 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.6] 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQC47tBjo0e9KBAAp/huYVA0Dq8PUGEMfBRNWg== 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout:[osd.7] 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQC77tBjbrvRJxAAWYRgsbZ8CbXMxfw1BLmIrQ== 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCi7tBjpOw8ERAAsJrwUmdFtrJ7Rdu7D5tpew== 2023-01-25T08:56:31.317 INFO:teuthology.orchestra.run.smithi008.stdout:[client.1] 2023-01-25T08:56:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCi7tBjIMvAFBAAgkqW5m08tMaFMzKnuHlIIQ== 2023-01-25T08:56:31.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T08:56:31.337 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T08:56:31.416 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.418 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.474 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.476 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.532 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.534 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.589 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.591 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.646 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T08:56:31.648 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T08:56:31.704 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.707 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.769 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.824 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.827 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.881 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.884 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T08:56:31.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T08:56:31.940 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T08:56:31.998 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T08:56:32.001 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T08:56:32.058 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T08:56:32.059 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T08:56:32.087 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T08:56:32.187 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T08:56:32.214 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T08:56:32.282 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T08:56:32.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T08:56:32.411 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T08:56:32.441 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T08:56:32.541 DEBUG:teuthology.orchestra.run.smithi008:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T08:56:32.570 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:32.572 DEBUG:teuthology.orchestra.run.smithi008:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T08:56:32.628 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T08:56:32.628 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T08:56:32.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T08:56:32.631 INFO:tasks.ceph.mon.a:Started 2023-01-25T08:56:32.632 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T08:56:32.632 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T08:56:32.634 INFO:tasks.ceph.mon.c:Started 2023-01-25T08:56:32.634 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T08:56:32.635 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T08:56:32.670 INFO:tasks.ceph.mon.b:Started 2023-01-25T08:56:32.670 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T08:56:32.671 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T08:56:32.673 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T08:56:32.675 INFO:tasks.ceph.mgr.y:Started 2023-01-25T08:56:32.676 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T08:56:32.676 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T08:56:32.678 INFO:tasks.ceph.mgr.x:Started 2023-01-25T08:56:32.679 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T08:56:32.679 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T08:56:33.066 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2023-01-25T08:56:33.082 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T08:56:33.082 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:33.083 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T08:56:33.111 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:33.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T08:56:33.180 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:33.181 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T08:56:33.249 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-01-25T08:56:33.249 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T08:56:33.317 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:33.318 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T08:56:33.348 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:33.349 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T08:56:33.418 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:33.419 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T08:56:33.489 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-01-25T08:56:33.489 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T08:56:33.559 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new d5589a5c-652d-41c4-9b5f-4440f7b52f45 0 2023-01-25T08:56:33.821 INFO:teuthology.orchestra.run.smithi008.stdout:0 2023-01-25T08:56:33.834 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new e5424e33-19e7-42c2-b7e7-de52c50d8aa7 1 2023-01-25T08:56:33.922 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:33.921+0000 7ffb895c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T08:56:33.940 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:33.940+0000 7fbbab936200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T08:56:33.996 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:33.994+0000 7ffb895c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T08:56:34.015 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:34.014+0000 7fbbab936200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T08:56:34.057 INFO:teuthology.orchestra.run.smithi008.stdout:1 2023-01-25T08:56:34.069 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new 29ac07f9-1427-47d4-af21-a70bcffb898e 2 2023-01-25T08:56:34.095 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:34.093+0000 7ffb895c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T08:56:34.213 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:34.212+0000 7fbbab936200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T08:56:34.288 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:34.286+0000 7ffb895c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T08:56:34.288 INFO:teuthology.orchestra.run.smithi008.stdout:2 2023-01-25T08:56:34.301 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new 35c680e5-76a0-4487-925b-5a02820a0c59 3 2023-01-25T08:56:34.410 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:34.409+0000 7fbbab936200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T08:56:34.520 INFO:teuthology.orchestra.run.smithi008.stdout:3 2023-01-25T08:56:34.533 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new b99283cb-7d71-47a4-a5f9-1e576f00d81b 4 2023-01-25T08:56:34.760 INFO:teuthology.orchestra.run.smithi008.stdout:4 2023-01-25T08:56:34.772 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new 737c7c42-b0c0-4cc7-9797-43b6ae7afb8c 5 2023-01-25T08:56:35.012 INFO:teuthology.orchestra.run.smithi008.stdout:5 2023-01-25T08:56:35.025 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new b8c3c2d2-84cb-465e-b559-94e344ead33b 6 2023-01-25T08:56:35.116 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:35.115+0000 7ffb895c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T08:56:35.253 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:35.252+0000 7fbbab936200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T08:56:35.263 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:35.261+0000 7ffb895c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T08:56:35.285 INFO:teuthology.orchestra.run.smithi008.stdout:6 2023-01-25T08:56:35.298 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --cluster ceph osd new 5addbd03-a232-4959-bf02-167a968e5f77 7 2023-01-25T08:56:35.401 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:35.400+0000 7fbbab936200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T08:56:35.496 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:35.495+0000 7ffb895c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T08:56:35.519 INFO:teuthology.orchestra.run.smithi008.stdout:7 2023-01-25T08:56:35.533 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T08:56:35.534 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T08:56:35.536 INFO:tasks.ceph.osd.0:Started 2023-01-25T08:56:35.536 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T08:56:35.536 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T08:56:35.538 INFO:tasks.ceph.osd.1:Started 2023-01-25T08:56:35.539 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T08:56:35.539 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T08:56:35.541 INFO:tasks.ceph.osd.2:Started 2023-01-25T08:56:35.541 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T08:56:35.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T08:56:35.544 INFO:tasks.ceph.osd.3:Started 2023-01-25T08:56:35.544 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T08:56:35.545 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T08:56:35.547 INFO:tasks.ceph.osd.4:Started 2023-01-25T08:56:35.548 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T08:56:35.548 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T08:56:35.550 INFO:tasks.ceph.osd.5:Started 2023-01-25T08:56:35.551 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T08:56:35.551 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T08:56:35.553 INFO:tasks.ceph.osd.6:Started 2023-01-25T08:56:35.554 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T08:56:35.554 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T08:56:35.556 INFO:tasks.ceph.osd.7:Started 2023-01-25T08:56:35.557 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T08:56:35.594 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:35.592+0000 7ffb895c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T08:56:35.649 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:35.648+0000 7fbbab936200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T08:56:35.727 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:35.726+0000 7fbbab936200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T08:56:35.757 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:35.755+0000 7ffb895c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T08:56:35.799 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:35.799 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"fsid":"a4dc1c97-bb3d-4eba-b23f-bf25be3327d8","created":"2023-01-25T08:56:32.950433+0000","modified":"2023-01-25T08:56:35.515489+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T08:56:35.515489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"d5589a5c-652d-41c4-9b5f-4440f7b52f45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5424e33-19e7-42c2-b7e7-de52c50d8aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"29ac07f9-1427-47d4-af21-a70bcffb898e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"35c680e5-76a0-4487-925b-5a02820a0c59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b99283cb-7d71-47a4-a5f9-1e576f00d81b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"737c7c42-b0c0-4cc7-9797-43b6ae7afb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8c3c2d2-84cb-465e-b559-94e344ead33b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5addbd03-a232-4959-bf02-167a968e5f77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T08:56:35.811 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T08:56:35.812 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T08:56:35.868 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:35.867+0000 7fbbab936200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T08:56:35.910 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:35.908+0000 7f4962237580 -1 Falling back to public interface 2023-01-25T08:56:36.039 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.037+0000 7ffb895c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T08:56:36.145 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.144+0000 7fbbab936200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T08:56:36.153 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:36.151+0000 7f1580f67580 -1 Falling back to public interface 2023-01-25T08:56:36.164 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:36.163+0000 7f35b1bc9580 -1 Falling back to public interface 2023-01-25T08:56:36.168 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:36.167+0000 7fd2805b2580 -1 Falling back to public interface 2023-01-25T08:56:36.170 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:36.168+0000 7f884e78b580 -1 Falling back to public interface 2023-01-25T08:56:36.173 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-01-25T08:56:36.171+0000 7fd5947df580 -1 Falling back to public interface 2023-01-25T08:56:36.181 INFO:tasks.ceph.osd.4.smithi008.stderr:2023-01-25T08:56:36.180+0000 7f1f4384c580 -1 Falling back to public interface 2023-01-25T08:56:36.185 INFO:tasks.ceph.osd.6.smithi008.stderr:2023-01-25T08:56:36.185+0000 7feae929d580 -1 Falling back to public interface 2023-01-25T08:56:36.232 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.230+0000 7ffb895c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T08:56:36.336 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.335+0000 7fbbab936200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T08:56:36.378 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.376+0000 7ffb895c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T08:56:36.482 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.482+0000 7fbbab936200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T08:56:36.498 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.496+0000 7ffb895c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T08:56:36.601 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.601+0000 7fbbab936200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T08:56:36.617 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T08:56:36.643 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.642+0000 7ffb895c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T08:56:36.728 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.726+0000 7ffb895c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T08:56:36.746 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.746+0000 7fbbab936200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T08:56:36.812 INFO:teuthology.misc.health.smithi006.stdout: 2023-01-25T08:56:36.813 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":10,"fsid":"a4dc1c97-bb3d-4eba-b23f-bf25be3327d8","created":"2023-01-25T08:56:32.950433+0000","modified":"2023-01-25T08:56:35.515489+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T08:56:35.515489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"d5589a5c-652d-41c4-9b5f-4440f7b52f45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5424e33-19e7-42c2-b7e7-de52c50d8aa7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"29ac07f9-1427-47d4-af21-a70bcffb898e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"35c680e5-76a0-4487-925b-5a02820a0c59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b99283cb-7d71-47a4-a5f9-1e576f00d81b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"737c7c42-b0c0-4cc7-9797-43b6ae7afb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b8c3c2d2-84cb-465e-b559-94e344ead33b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5addbd03-a232-4959-bf02-167a968e5f77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T08:56:36.825 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T08:56:36.828 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:36.827+0000 7fbbab936200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T08:56:36.956 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:36.955+0000 7ffb895c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T08:56:37.038 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:37.036+0000 7ffb895c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T08:56:37.054 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:37.054+0000 7fbbab936200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T08:56:37.135 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:37.134+0000 7fbbab936200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T08:56:37.687 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:37.685+0000 7ffb895c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T08:56:37.778 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:37.777+0000 7fbbab936200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T08:56:37.952 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:37.951+0000 7ffb895c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T08:56:38.030 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:38.028+0000 7ffb895c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T08:56:38.042 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.041+0000 7fbbab936200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T08:56:38.115 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:38.113+0000 7ffb895c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T08:56:38.120 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.120+0000 7fbbab936200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T08:56:38.206 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.205+0000 7fbbab936200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T08:56:38.305 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:38.303+0000 7ffb895c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T08:56:38.381 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:38.379+0000 7ffb895c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T08:56:38.398 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.397+0000 7fbbab936200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T08:56:38.472 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.471+0000 7fbbab936200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T08:56:38.538 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:38.536+0000 7ffb895c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T08:56:38.637 INFO:tasks.ceph.mgr.x.smithi008.stderr:2023-01-25T08:56:38.636+0000 7fbbab936200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T08:56:39.421 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:39.419+0000 7f4962237580 -1 osd.2 0 log_to_monitors true 2023-01-25T08:56:39.562 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:39.560+0000 7ffb43139700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T08:56:39.563 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-01-25T08:56:39.561+0000 7ffb43139700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T08:56:39.632 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:39.631+0000 7fd2805b2580 -1 osd.7 0 log_to_monitors true 2023-01-25T08:56:39.652 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:39.651+0000 7f1580f67580 -1 osd.3 0 log_to_monitors true 2023-01-25T08:56:39.663 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:39.663+0000 7f35b1bc9580 -1 osd.5 0 log_to_monitors true 2023-01-25T08:56:39.683 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:39.682+0000 7f884e78b580 -1 osd.1 0 log_to_monitors true 2023-01-25T08:56:39.689 INFO:tasks.ceph.osd.4.smithi008.stderr:2023-01-25T08:56:39.688+0000 7f1f4384c580 -1 osd.4 0 log_to_monitors true 2023-01-25T08:56:39.697 INFO:tasks.ceph.osd.6.smithi008.stderr:2023-01-25T08:56:39.696+0000 7feae929d580 -1 osd.6 0 log_to_monitors true 2023-01-25T08:56:39.699 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-01-25T08:56:39.698+0000 7fd5947df580 -1 osd.0 0 log_to_monitors true 2023-01-25T08:56:40.751 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:40.749+0000 7f49565df700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T08:56:40.752 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:40.751+0000 7fd27495a700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T08:56:40.756 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:40.755+0000 7f157530f700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T08:56:40.757 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:40.756+0000 7f35a5f71700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T08:56:40.757 INFO:tasks.ceph.osd.4.smithi008.stderr:2023-01-25T08:56:40.756+0000 7f1f37bf4700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T08:56:40.758 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:40.756+0000 7f359d408700 -1 osd.5 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.759 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:40.756+0000 7f156c7a6700 -1 osd.3 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.759 INFO:tasks.ceph.osd.6.smithi008.stderr:2023-01-25T08:56:40.757+0000 7feadd645700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T08:56:40.759 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-01-25T08:56:40.757+0000 7fd588b87700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T08:56:40.760 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:40.757+0000 7fd26bdf1700 -1 osd.7 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.760 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:40.757+0000 7f494da76700 -1 osd.2 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.761 INFO:tasks.ceph.osd.4.smithi008.stderr:2023-01-25T08:56:40.757+0000 7f1f2f08b700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.761 INFO:tasks.ceph.osd.6.smithi008.stderr:2023-01-25T08:56:40.758+0000 7fead4adc700 -1 osd.6 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.761 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:40.757+0000 7f8842b33700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T08:56:40.761 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:40.757+0000 7f8839fca700 -1 osd.1 0 handle_osd_map injecting map crc failure 2023-01-25T08:56:40.762 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:40.760+0000 7f1570ef1700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.762 INFO:tasks.ceph.osd.4.smithi008.stderr:2023-01-25T08:56:40.761+0000 7f1f337d6700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.763 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:40.761+0000 7fd27053c700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.764 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:40.762+0000 7f883e715700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.764 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:40.762+0000 7f35a1b53700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.764 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-01-25T08:56:40.762+0000 7fd584769700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.765 INFO:tasks.ceph.osd.6.smithi008.stderr:2023-01-25T08:56:40.763+0000 7fead9227700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:40.765 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:40.762+0000 7f49521c1700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T08:56:41.752 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:41.750+0000 7f156c7a6700 -1 osd.3 12 handle_osd_map injecting map crc failure 2023-01-25T08:56:41.752 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:41.751+0000 7f156c7a6700 -1 osd.3 12 handle_osd_map injecting map crc failure 2023-01-25T08:56:41.754 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:41.752+0000 7f494da76700 -1 osd.2 12 handle_osd_map injecting map crc failure 2023-01-25T08:56:43.632 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T08:56:43.969 INFO:teuthology.misc.health.smithi006.stdout: 2023-01-25T08:56:43.969 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":15,"fsid":"a4dc1c97-bb3d-4eba-b23f-bf25be3327d8","created":"2023-01-25T08:56:32.950433+0000","modified":"2023-01-25T08:56:43.765847+0000","last_up_change":"2023-01-25T08:56:41.749597+0000","last_in_change":"2023-01-25T08:56:35.515489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T08:56:42.612528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"d5589a5c-652d-41c4-9b5f-4440f7b52f45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6812","nonce":106336}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6813","nonce":106336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6815","nonce":106336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6814","nonce":106336}]},"public_addr":"172.21.15.6:6812/106336","cluster_addr":"172.21.15.6:6813/106336","heartbeat_back_addr":"172.21.15.6:6815/106336","heartbeat_front_addr":"172.21.15.6:6814/106336","state":["exists","up"]},{"osd":1,"uuid":"e5424e33-19e7-42c2-b7e7-de52c50d8aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6808","nonce":106335}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6809","nonce":106335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6811","nonce":106335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6810","nonce":106335}]},"public_addr":"172.21.15.6:6808/106335","cluster_addr":"172.21.15.6:6809/106335","heartbeat_back_addr":"172.21.15.6:6811/106335","heartbeat_front_addr":"172.21.15.6:6810/106335","state":["exists","up"]},{"osd":2,"uuid":"29ac07f9-1427-47d4-af21-a70bcffb898e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6800","nonce":106333}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6801","nonce":106333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6803","nonce":106333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6802","nonce":106333}]},"public_addr":"172.21.15.6:6800/106333","cluster_addr":"172.21.15.6:6801/106333","heartbeat_back_addr":"172.21.15.6:6803/106333","heartbeat_front_addr":"172.21.15.6:6802/106333","state":["exists","up"]},{"osd":3,"uuid":"35c680e5-76a0-4487-925b-5a02820a0c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6804","nonce":106334}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6805","nonce":106334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6807","nonce":106334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6806","nonce":106334}]},"public_addr":"172.21.15.6:6804/106334","cluster_addr":"172.21.15.6:6805/106334","heartbeat_back_addr":"172.21.15.6:6807/106334","heartbeat_front_addr":"172.21.15.6:6806/106334","state":["exists","up"]},{"osd":4,"uuid":"b99283cb-7d71-47a4-a5f9-1e576f00d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6808","nonce":106245}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6809","nonce":106245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6811","nonce":106245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6810","nonce":106245}]},"public_addr":"172.21.15.8:6808/106245","cluster_addr":"172.21.15.8:6809/106245","heartbeat_back_addr":"172.21.15.8:6811/106245","heartbeat_front_addr":"172.21.15.8:6810/106245","state":["exists","up"]},{"osd":5,"uuid":"737c7c42-b0c0-4cc7-9797-43b6ae7afb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6800","nonce":106235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6801","nonce":106235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6803","nonce":106235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6802","nonce":106235}]},"public_addr":"172.21.15.8:6800/106235","cluster_addr":"172.21.15.8:6801/106235","heartbeat_back_addr":"172.21.15.8:6803/106235","heartbeat_front_addr":"172.21.15.8:6802/106235","state":["exists","up"]},{"osd":6,"uuid":"b8c3c2d2-84cb-465e-b559-94e344ead33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6812","nonce":106251}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6813","nonce":106251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6815","nonce":106251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6814","nonce":106251}]},"public_addr":"172.21.15.8:6812/106251","cluster_addr":"172.21.15.8:6813/106251","heartbeat_back_addr":"172.21.15.8:6815/106251","heartbeat_front_addr":"172.21.15.8:6814/106251","state":["exists","up"]},{"osd":7,"uuid":"5addbd03-a232-4959-bf02-167a968e5f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6804","nonce":106236}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6805","nonce":106236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6807","nonce":106236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6806","nonce":106236}]},"public_addr":"172.21.15.8:6804/106236","cluster_addr":"172.21.15.8:6805/106236","heartbeat_back_addr":"172.21.15.8:6807/106236","heartbeat_front_addr":"172.21.15.8:6806/106236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.652766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.667771+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.414327+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.685351+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.641553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.664978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.675433+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.674607+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T08:56:43.981 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T08:56:43.982 INFO:tasks.ceph:Creating RBD pool 2023-01-25T08:56:43.982 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T08:56:44.235 INFO:tasks.ceph.osd.7.smithi008.stderr:2023-01-25T08:56:44.234+0000 7fd269ded700 -1 osd.7 13 handle_osd_map injecting map crc failure 2023-01-25T08:56:44.779 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:44.778+0000 7f156c7a6700 -1 osd.3 15 handle_osd_map injecting map crc failure 2023-01-25T08:56:44.789 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'rbd' created 2023-01-25T08:56:44.797 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T08:56:45.818 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-01-25T08:56:45.816+0000 7f156c7a6700 -1 osd.3 16 handle_osd_map injecting map crc failure 2023-01-25T08:56:45.819 INFO:tasks.ceph.osd.5.smithi008.stderr:2023-01-25T08:56:45.819+0000 7f359d408700 -1 osd.5 16 handle_osd_map injecting map crc failure 2023-01-25T08:56:45.823 INFO:teuthology.orchestra.run.smithi006.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T08:56:45.838 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T08:56:45.839 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T08:56:45.839 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T08:56:45.840 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T08:56:45.841 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T08:56:46.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:46.257 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":4,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6816","nonce":106035}]},"active_addr":"172.21.15.6:6816/106035","active_change":"2023-01-25T08:56:38.551140+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":4163353710}]},{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":236493780}]},{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":968219941}]}]}} 2023-01-25T08:56:46.262 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T08:56:46.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T08:56:46.262 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T08:56:46.593 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:46.593 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":17,"fsid":"a4dc1c97-bb3d-4eba-b23f-bf25be3327d8","created":"2023-01-25T08:56:32.950433+0000","modified":"2023-01-25T08:56:45.778792+0000","last_up_change":"2023-01-25T08:56:41.749597+0000","last_in_change":"2023-01-25T08:56:35.515489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T08:56:42.612528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T08:56:44.337573+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d5589a5c-652d-41c4-9b5f-4440f7b52f45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6812","nonce":106336}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6813","nonce":106336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6815","nonce":106336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6814","nonce":106336}]},"public_addr":"172.21.15.6:6812/106336","cluster_addr":"172.21.15.6:6813/106336","heartbeat_back_addr":"172.21.15.6:6815/106336","heartbeat_front_addr":"172.21.15.6:6814/106336","state":["exists","up"]},{"osd":1,"uuid":"e5424e33-19e7-42c2-b7e7-de52c50d8aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6808","nonce":106335}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6809","nonce":106335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6811","nonce":106335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6810","nonce":106335}]},"public_addr":"172.21.15.6:6808/106335","cluster_addr":"172.21.15.6:6809/106335","heartbeat_back_addr":"172.21.15.6:6811/106335","heartbeat_front_addr":"172.21.15.6:6810/106335","state":["exists","up"]},{"osd":2,"uuid":"29ac07f9-1427-47d4-af21-a70bcffb898e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6800","nonce":106333}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6801","nonce":106333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6803","nonce":106333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6802","nonce":106333}]},"public_addr":"172.21.15.6:6800/106333","cluster_addr":"172.21.15.6:6801/106333","heartbeat_back_addr":"172.21.15.6:6803/106333","heartbeat_front_addr":"172.21.15.6:6802/106333","state":["exists","up"]},{"osd":3,"uuid":"35c680e5-76a0-4487-925b-5a02820a0c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6804","nonce":106334}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6805","nonce":106334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6807","nonce":106334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6806","nonce":106334}]},"public_addr":"172.21.15.6:6804/106334","cluster_addr":"172.21.15.6:6805/106334","heartbeat_back_addr":"172.21.15.6:6807/106334","heartbeat_front_addr":"172.21.15.6:6806/106334","state":["exists","up"]},{"osd":4,"uuid":"b99283cb-7d71-47a4-a5f9-1e576f00d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6808","nonce":106245}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6809","nonce":106245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6811","nonce":106245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6810","nonce":106245}]},"public_addr":"172.21.15.8:6808/106245","cluster_addr":"172.21.15.8:6809/106245","heartbeat_back_addr":"172.21.15.8:6811/106245","heartbeat_front_addr":"172.21.15.8:6810/106245","state":["exists","up"]},{"osd":5,"uuid":"737c7c42-b0c0-4cc7-9797-43b6ae7afb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6800","nonce":106235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6801","nonce":106235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6803","nonce":106235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6802","nonce":106235}]},"public_addr":"172.21.15.8:6800/106235","cluster_addr":"172.21.15.8:6801/106235","heartbeat_back_addr":"172.21.15.8:6803/106235","heartbeat_front_addr":"172.21.15.8:6802/106235","state":["exists","up"]},{"osd":6,"uuid":"b8c3c2d2-84cb-465e-b559-94e344ead33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6812","nonce":106251}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6813","nonce":106251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6815","nonce":106251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6814","nonce":106251}]},"public_addr":"172.21.15.8:6812/106251","cluster_addr":"172.21.15.8:6813/106251","heartbeat_back_addr":"172.21.15.8:6815/106251","heartbeat_front_addr":"172.21.15.8:6814/106251","state":["exists","up"]},{"osd":7,"uuid":"5addbd03-a232-4959-bf02-167a968e5f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6804","nonce":106236}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6805","nonce":106236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6807","nonce":106236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6806","nonce":106236}]},"public_addr":"172.21.15.8:6804/106236","cluster_addr":"172.21.15.8:6805/106236","heartbeat_back_addr":"172.21.15.8:6807/106236","heartbeat_front_addr":"172.21.15.8:6806/106236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.652766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.667771+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.414327+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.685351+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.641553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.664978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.675433+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.674607+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T08:56:46.607 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T08:56:46.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T08:56:46.939 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:46.939 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":18,"fsid":"a4dc1c97-bb3d-4eba-b23f-bf25be3327d8","created":"2023-01-25T08:56:32.950433+0000","modified":"2023-01-25T08:56:46.817437+0000","last_up_change":"2023-01-25T08:56:41.749597+0000","last_in_change":"2023-01-25T08:56:35.515489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T08:56:42.612528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T08:56:44.337573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"d5589a5c-652d-41c4-9b5f-4440f7b52f45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6812","nonce":106336}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6813","nonce":106336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6815","nonce":106336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6814","nonce":106336}]},"public_addr":"172.21.15.6:6812/106336","cluster_addr":"172.21.15.6:6813/106336","heartbeat_back_addr":"172.21.15.6:6815/106336","heartbeat_front_addr":"172.21.15.6:6814/106336","state":["exists","up"]},{"osd":1,"uuid":"e5424e33-19e7-42c2-b7e7-de52c50d8aa7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6808","nonce":106335}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6809","nonce":106335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6811","nonce":106335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6810","nonce":106335}]},"public_addr":"172.21.15.6:6808/106335","cluster_addr":"172.21.15.6:6809/106335","heartbeat_back_addr":"172.21.15.6:6811/106335","heartbeat_front_addr":"172.21.15.6:6810/106335","state":["exists","up"]},{"osd":2,"uuid":"29ac07f9-1427-47d4-af21-a70bcffb898e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6800","nonce":106333}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6801","nonce":106333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6803","nonce":106333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6802","nonce":106333}]},"public_addr":"172.21.15.6:6800/106333","cluster_addr":"172.21.15.6:6801/106333","heartbeat_back_addr":"172.21.15.6:6803/106333","heartbeat_front_addr":"172.21.15.6:6802/106333","state":["exists","up"]},{"osd":3,"uuid":"35c680e5-76a0-4487-925b-5a02820a0c59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6804","nonce":106334}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6805","nonce":106334}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6807","nonce":106334}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.6:6806","nonce":106334}]},"public_addr":"172.21.15.6:6804/106334","cluster_addr":"172.21.15.6:6805/106334","heartbeat_back_addr":"172.21.15.6:6807/106334","heartbeat_front_addr":"172.21.15.6:6806/106334","state":["exists","up"]},{"osd":4,"uuid":"b99283cb-7d71-47a4-a5f9-1e576f00d81b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6808","nonce":106245}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6809","nonce":106245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6811","nonce":106245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6810","nonce":106245}]},"public_addr":"172.21.15.8:6808/106245","cluster_addr":"172.21.15.8:6809/106245","heartbeat_back_addr":"172.21.15.8:6811/106245","heartbeat_front_addr":"172.21.15.8:6810/106245","state":["exists","up"]},{"osd":5,"uuid":"737c7c42-b0c0-4cc7-9797-43b6ae7afb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6800","nonce":106235}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6801","nonce":106235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6803","nonce":106235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6802","nonce":106235}]},"public_addr":"172.21.15.8:6800/106235","cluster_addr":"172.21.15.8:6801/106235","heartbeat_back_addr":"172.21.15.8:6803/106235","heartbeat_front_addr":"172.21.15.8:6802/106235","state":["exists","up"]},{"osd":6,"uuid":"b8c3c2d2-84cb-465e-b559-94e344ead33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6812","nonce":106251}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6813","nonce":106251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6815","nonce":106251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6814","nonce":106251}]},"public_addr":"172.21.15.8:6812/106251","cluster_addr":"172.21.15.8:6813/106251","heartbeat_back_addr":"172.21.15.8:6815/106251","heartbeat_front_addr":"172.21.15.8:6814/106251","state":["exists","up"]},{"osd":7,"uuid":"5addbd03-a232-4959-bf02-167a968e5f77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6804","nonce":106236}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6805","nonce":106236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6807","nonce":106236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.8:6806","nonce":106236}]},"public_addr":"172.21.15.8:6804/106236","cluster_addr":"172.21.15.8:6805/106236","heartbeat_back_addr":"172.21.15.8:6807/106236","heartbeat_front_addr":"172.21.15.8:6806/106236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.652766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.667771+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.414327+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.685351+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.641553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.664978+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.675433+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T08:56:40.674607+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T08:56:46.953 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T08:56:46.954 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T08:56:46.954 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T08:56:46.955 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T08:56:46.956 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T08:56:46.956 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T08:56:46.957 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T08:56:46.957 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T08:56:47.237 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.237 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T08:56:47.257 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.257 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T08:56:47.260 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.260 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2023-01-25T08:56:47.268 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T08:56:47.274 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:47.274 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T08:56:47.275 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T08:56:47.277 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.277 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T08:56:47.289 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T08:56:47.647 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-01-25T08:56:47.640+0000 7f494ba72700 -1 osd.2 17 handle_osd_map injecting map crc failure 2023-01-25T08:56:47.917 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:47.921 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:47.922 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:47.922 INFO:teuthology.orchestra.run.smithi006.stdout:55834574850 2023-01-25T08:56:47.934 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:47.934 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:47.944 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:47.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-01-25T08:56:47.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.5 2023-01-25T08:56:47.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T08:56:47.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2023-01-25T08:56:47.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2023-01-25T08:56:47.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2023-01-25T08:56:47.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.7 2023-01-25T08:56:47.996 INFO:teuthology.orchestra.run.smithi006.stdout:55834574849 2023-01-25T08:56:48.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-01-25T08:56:48.945 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T08:56:48.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T08:56:48.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T08:56:48.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T08:56:48.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T08:56:48.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T08:56:48.972 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T08:56:49.012 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T08:56:49.576 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.592 INFO:teuthology.orchestra.run.smithi006.stdout:55834574853 2023-01-25T08:56:49.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.5 2023-01-25T08:56:49.596 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.605 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.6 2023-01-25T08:56:49.615 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.2 2023-01-25T08:56:49.620 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.630 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.634 INFO:teuthology.orchestra.run.smithi006.stdout:55834574852 2023-01-25T08:56:49.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.7 2023-01-25T08:56:49.647 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.648 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.650 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2023-01-25T08:56:49.652 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.0 2023-01-25T08:56:49.666 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.4 2023-01-25T08:56:49.669 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.709 INFO:teuthology.orchestra.run.smithi006.stdout:55834574851 2023-01-25T08:56:49.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.1 2023-01-25T08:56:49.723 DEBUG:teuthology.parallel:result is None 2023-01-25T08:56:49.724 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T08:56:49.724 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T08:56:50.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:50.096 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-01-25T08:56:50.107 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T08:56:48.556581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41472,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754933248,"statfs":{"total":773094113280,"available":773051645952,"internally_reserved":0,"allocated":1085440,"data_stored":1410779,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827927+0000","last_change":"2023-01-25T08:56:46.542968+0000","last_active":"2023-01-25T08:56:46.827927+0000","last_peered":"2023-01-25T08:56:46.827927+0000","last_clean":"2023-01-25T08:56:46.827927+0000","last_became_active":"2023-01-25T08:56:46.542722+0000","last_became_peered":"2023-01-25T08:56:46.542722+0000","last_unstale":"2023-01-25T08:56:46.827927+0000","last_undegraded":"2023-01-25T08:56:46.827927+0000","last_fullsized":"2023-01-25T08:56:46.827927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:57.589071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.829553+0000","last_change":"2023-01-25T08:56:46.540445+0000","last_active":"2023-01-25T08:56:46.829553+0000","last_peered":"2023-01-25T08:56:46.829553+0000","last_clean":"2023-01-25T08:56:46.829553+0000","last_became_active":"2023-01-25T08:56:46.540172+0000","last_became_peered":"2023-01-25T08:56:46.540172+0000","last_unstale":"2023-01-25T08:56:46.829553+0000","last_undegraded":"2023-01-25T08:56:46.829553+0000","last_fullsized":"2023-01-25T08:56:46.829553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.284924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827870+0000","last_change":"2023-01-25T08:56:45.823843+0000","last_active":"2023-01-25T08:56:46.827870+0000","last_peered":"2023-01-25T08:56:46.827870+0000","last_clean":"2023-01-25T08:56:46.827870+0000","last_became_active":"2023-01-25T08:56:45.823567+0000","last_became_peered":"2023-01-25T08:56:45.823567+0000","last_unstale":"2023-01-25T08:56:46.827870+0000","last_undegraded":"2023-01-25T08:56:46.827870+0000","last_fullsized":"2023-01-25T08:56:46.827870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:02.016887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827781+0000","last_change":"2023-01-25T08:56:45.827804+0000","last_active":"2023-01-25T08:56:46.827781+0000","last_peered":"2023-01-25T08:56:46.827781+0000","last_clean":"2023-01-25T08:56:46.827781+0000","last_became_active":"2023-01-25T08:56:45.827629+0000","last_became_peered":"2023-01-25T08:56:45.827629+0000","last_unstale":"2023-01-25T08:56:46.827781+0000","last_undegraded":"2023-01-25T08:56:46.827781+0000","last_fullsized":"2023-01-25T08:56:46.827781+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.572338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.251165+0000","last_change":"2023-01-25T08:56:44.571864+0000","last_active":"2023-01-25T08:56:47.251165+0000","last_peered":"2023-01-25T08:56:47.251165+0000","last_clean":"2023-01-25T08:56:47.251165+0000","last_became_active":"2023-01-25T08:56:44.570852+0000","last_became_peered":"2023-01-25T08:56:44.570852+0000","last_unstale":"2023-01-25T08:56:47.251165+0000","last_undegraded":"2023-01-25T08:56:47.251165+0000","last_fullsized":"2023-01-25T08:56:47.251165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_clean_scrub_stamp":"2023-01-25T08:56:42.760430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:47.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827790+0000","last_change":"2023-01-25T08:56:45.824516+0000","last_active":"2023-01-25T08:56:46.827790+0000","last_peered":"2023-01-25T08:56:46.827790+0000","last_clean":"2023-01-25T08:56:46.827790+0000","last_became_active":"2023-01-25T08:56:45.824228+0000","last_became_peered":"2023-01-25T08:56:45.824228+0000","last_unstale":"2023-01-25T08:56:46.827790+0000","last_undegraded":"2023-01-25T08:56:46.827790+0000","last_fullsized":"2023-01-25T08:56:46.827790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:51.431839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.826288+0000","last_change":"2023-01-25T08:56:45.823862+0000","last_active":"2023-01-25T08:56:46.826288+0000","last_peered":"2023-01-25T08:56:46.826288+0000","last_clean":"2023-01-25T08:56:46.826288+0000","last_became_active":"2023-01-25T08:56:45.823566+0000","last_became_peered":"2023-01-25T08:56:45.823566+0000","last_unstale":"2023-01-25T08:56:46.826288+0000","last_undegraded":"2023-01-25T08:56:46.826288+0000","last_fullsized":"2023-01-25T08:56:46.826288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:01.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827780+0000","last_change":"2023-01-25T08:56:45.824420+0000","last_active":"2023-01-25T08:56:46.827780+0000","last_peered":"2023-01-25T08:56:46.827780+0000","last_clean":"2023-01-25T08:56:46.827780+0000","last_became_active":"2023-01-25T08:56:45.824209+0000","last_became_peered":"2023-01-25T08:56:45.824209+0000","last_unstale":"2023-01-25T08:56:46.827780+0000","last_undegraded":"2023-01-25T08:56:46.827780+0000","last_fullsized":"2023-01-25T08:56:46.827780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:09.529598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.219870+0000","last_change":"2023-01-25T08:56:45.823947+0000","last_active":"2023-01-25T08:56:47.219870+0000","last_peered":"2023-01-25T08:56:47.219870+0000","last_clean":"2023-01-25T08:56:47.219870+0000","last_became_active":"2023-01-25T08:56:45.823652+0000","last_became_peered":"2023-01-25T08:56:45.823652+0000","last_unstale":"2023-01-25T08:56:47.219870+0000","last_undegraded":"2023-01-25T08:56:47.219870+0000","last_fullsized":"2023-01-25T08:56:47.219870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:53.603934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":43720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.8319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":43195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5552,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366288,"statfs":{"total":96636764160,"available":96631078912,"internally_reserved":0,"allocated":143360,"data_stored":567072,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":44309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":42457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":151552,"data_stored":571700,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T08:56:50.109 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T08:56:50.437 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:50.439 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-01-25T08:56:50.450 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T08:56:48.556581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41472,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754933248,"statfs":{"total":773094113280,"available":773051645952,"internally_reserved":0,"allocated":1085440,"data_stored":1410779,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827927+0000","last_change":"2023-01-25T08:56:46.542968+0000","last_active":"2023-01-25T08:56:46.827927+0000","last_peered":"2023-01-25T08:56:46.827927+0000","last_clean":"2023-01-25T08:56:46.827927+0000","last_became_active":"2023-01-25T08:56:46.542722+0000","last_became_peered":"2023-01-25T08:56:46.542722+0000","last_unstale":"2023-01-25T08:56:46.827927+0000","last_undegraded":"2023-01-25T08:56:46.827927+0000","last_fullsized":"2023-01-25T08:56:46.827927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:57.589071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.829553+0000","last_change":"2023-01-25T08:56:46.540445+0000","last_active":"2023-01-25T08:56:46.829553+0000","last_peered":"2023-01-25T08:56:46.829553+0000","last_clean":"2023-01-25T08:56:46.829553+0000","last_became_active":"2023-01-25T08:56:46.540172+0000","last_became_peered":"2023-01-25T08:56:46.540172+0000","last_unstale":"2023-01-25T08:56:46.829553+0000","last_undegraded":"2023-01-25T08:56:46.829553+0000","last_fullsized":"2023-01-25T08:56:46.829553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.284924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827870+0000","last_change":"2023-01-25T08:56:45.823843+0000","last_active":"2023-01-25T08:56:46.827870+0000","last_peered":"2023-01-25T08:56:46.827870+0000","last_clean":"2023-01-25T08:56:46.827870+0000","last_became_active":"2023-01-25T08:56:45.823567+0000","last_became_peered":"2023-01-25T08:56:45.823567+0000","last_unstale":"2023-01-25T08:56:46.827870+0000","last_undegraded":"2023-01-25T08:56:46.827870+0000","last_fullsized":"2023-01-25T08:56:46.827870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:02.016887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827781+0000","last_change":"2023-01-25T08:56:45.827804+0000","last_active":"2023-01-25T08:56:46.827781+0000","last_peered":"2023-01-25T08:56:46.827781+0000","last_clean":"2023-01-25T08:56:46.827781+0000","last_became_active":"2023-01-25T08:56:45.827629+0000","last_became_peered":"2023-01-25T08:56:45.827629+0000","last_unstale":"2023-01-25T08:56:46.827781+0000","last_undegraded":"2023-01-25T08:56:46.827781+0000","last_fullsized":"2023-01-25T08:56:46.827781+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.572338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.251165+0000","last_change":"2023-01-25T08:56:44.571864+0000","last_active":"2023-01-25T08:56:47.251165+0000","last_peered":"2023-01-25T08:56:47.251165+0000","last_clean":"2023-01-25T08:56:47.251165+0000","last_became_active":"2023-01-25T08:56:44.570852+0000","last_became_peered":"2023-01-25T08:56:44.570852+0000","last_unstale":"2023-01-25T08:56:47.251165+0000","last_undegraded":"2023-01-25T08:56:47.251165+0000","last_fullsized":"2023-01-25T08:56:47.251165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_clean_scrub_stamp":"2023-01-25T08:56:42.760430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:47.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827790+0000","last_change":"2023-01-25T08:56:45.824516+0000","last_active":"2023-01-25T08:56:46.827790+0000","last_peered":"2023-01-25T08:56:46.827790+0000","last_clean":"2023-01-25T08:56:46.827790+0000","last_became_active":"2023-01-25T08:56:45.824228+0000","last_became_peered":"2023-01-25T08:56:45.824228+0000","last_unstale":"2023-01-25T08:56:46.827790+0000","last_undegraded":"2023-01-25T08:56:46.827790+0000","last_fullsized":"2023-01-25T08:56:46.827790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:51.431839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.826288+0000","last_change":"2023-01-25T08:56:45.823862+0000","last_active":"2023-01-25T08:56:46.826288+0000","last_peered":"2023-01-25T08:56:46.826288+0000","last_clean":"2023-01-25T08:56:46.826288+0000","last_became_active":"2023-01-25T08:56:45.823566+0000","last_became_peered":"2023-01-25T08:56:45.823566+0000","last_unstale":"2023-01-25T08:56:46.826288+0000","last_undegraded":"2023-01-25T08:56:46.826288+0000","last_fullsized":"2023-01-25T08:56:46.826288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:01.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827780+0000","last_change":"2023-01-25T08:56:45.824420+0000","last_active":"2023-01-25T08:56:46.827780+0000","last_peered":"2023-01-25T08:56:46.827780+0000","last_clean":"2023-01-25T08:56:46.827780+0000","last_became_active":"2023-01-25T08:56:45.824209+0000","last_became_peered":"2023-01-25T08:56:45.824209+0000","last_unstale":"2023-01-25T08:56:46.827780+0000","last_undegraded":"2023-01-25T08:56:46.827780+0000","last_fullsized":"2023-01-25T08:56:46.827780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:09.529598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.219870+0000","last_change":"2023-01-25T08:56:45.823947+0000","last_active":"2023-01-25T08:56:47.219870+0000","last_peered":"2023-01-25T08:56:47.219870+0000","last_clean":"2023-01-25T08:56:47.219870+0000","last_became_active":"2023-01-25T08:56:45.823652+0000","last_became_peered":"2023-01-25T08:56:45.823652+0000","last_unstale":"2023-01-25T08:56:47.219870+0000","last_undegraded":"2023-01-25T08:56:47.219870+0000","last_fullsized":"2023-01-25T08:56:47.219870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:53.603934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":43720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.8319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":43195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5552,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366288,"statfs":{"total":96636764160,"available":96631078912,"internally_reserved":0,"allocated":143360,"data_stored":567072,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":44309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":42457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":151552,"data_stored":571700,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T08:56:50.451 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T08:56:50.452 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T08:56:50.452 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T08:56:50.452 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T08:56:50.821 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:50.821 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T08:56:50.834 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T08:56:50.835 INFO:teuthology.run_tasks:Running task exec... 2023-01-25T08:56:50.846 INFO:teuthology.task.exec:Executing custom commands... 2023-01-25T08:56:50.847 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi006.front.sepia.ceph.com 2023-01-25T08:56:50.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2023-01-25T08:56:51.172 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-01-25T08:56:51.172 INFO:teuthology.orchestra.run.smithi006.stdout: "active": true, 2023-01-25T08:56:51.173 INFO:teuthology.orchestra.run.smithi006.stdout: "last_optimize_duration": "0:00:00.000528", 2023-01-25T08:56:51.173 INFO:teuthology.orchestra.run.smithi006.stdout: "last_optimize_started": "Wed Jan 25 08:56:38 2023", 2023-01-25T08:56:51.173 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "upmap", 2023-01-25T08:56:51.173 INFO:teuthology.orchestra.run.smithi006.stdout: "no_optimization_needed": false, 2023-01-25T08:56:51.173 INFO:teuthology.orchestra.run.smithi006.stdout: "optimize_result": "No pools available", 2023-01-25T08:56:51.174 INFO:teuthology.orchestra.run.smithi006.stdout: "plans": [] 2023-01-25T08:56:51.174 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-01-25T08:56:51.186 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2023-01-25T08:56:51.927 DEBUG:teuthology.orchestra.run.smithi006:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2023-01-25T08:56:52.224 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-01-25T08:56:52.223+0000 7f8837fc6700 -1 osd.1 18 handle_osd_map injecting map crc failure 2023-01-25T08:56:52.274 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-25T08:56:52.302 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T08:56:52.303 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri6-testing-2023-01-24-0746'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 3, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2023-01-24-0746', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T08:56:52.303 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2023-01-25T08:56:52.303 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-25T08:56:52.304 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T08:56:52.304 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T08:56:52.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:52.632 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-01-25T08:56:52.644 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-01-25T08:56:52.557208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41472,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754933248,"statfs":{"total":773094113280,"available":773051645952,"internally_reserved":0,"allocated":1085440,"data_stored":1410779,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000627"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827927+0000","last_change":"2023-01-25T08:56:46.542968+0000","last_active":"2023-01-25T08:56:46.827927+0000","last_peered":"2023-01-25T08:56:46.827927+0000","last_clean":"2023-01-25T08:56:46.827927+0000","last_became_active":"2023-01-25T08:56:46.542722+0000","last_became_peered":"2023-01-25T08:56:46.542722+0000","last_unstale":"2023-01-25T08:56:46.827927+0000","last_undegraded":"2023-01-25T08:56:46.827927+0000","last_fullsized":"2023-01-25T08:56:46.827927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:57.589071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.829553+0000","last_change":"2023-01-25T08:56:46.540445+0000","last_active":"2023-01-25T08:56:46.829553+0000","last_peered":"2023-01-25T08:56:46.829553+0000","last_clean":"2023-01-25T08:56:46.829553+0000","last_became_active":"2023-01-25T08:56:46.540172+0000","last_became_peered":"2023-01-25T08:56:46.540172+0000","last_unstale":"2023-01-25T08:56:46.829553+0000","last_undegraded":"2023-01-25T08:56:46.829553+0000","last_fullsized":"2023-01-25T08:56:46.829553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.284924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827870+0000","last_change":"2023-01-25T08:56:45.823843+0000","last_active":"2023-01-25T08:56:46.827870+0000","last_peered":"2023-01-25T08:56:46.827870+0000","last_clean":"2023-01-25T08:56:46.827870+0000","last_became_active":"2023-01-25T08:56:45.823567+0000","last_became_peered":"2023-01-25T08:56:45.823567+0000","last_unstale":"2023-01-25T08:56:46.827870+0000","last_undegraded":"2023-01-25T08:56:46.827870+0000","last_fullsized":"2023-01-25T08:56:46.827870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:02.016887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827781+0000","last_change":"2023-01-25T08:56:45.827804+0000","last_active":"2023-01-25T08:56:46.827781+0000","last_peered":"2023-01-25T08:56:46.827781+0000","last_clean":"2023-01-25T08:56:46.827781+0000","last_became_active":"2023-01-25T08:56:45.827629+0000","last_became_peered":"2023-01-25T08:56:45.827629+0000","last_unstale":"2023-01-25T08:56:46.827781+0000","last_undegraded":"2023-01-25T08:56:46.827781+0000","last_fullsized":"2023-01-25T08:56:46.827781+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.572338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.251165+0000","last_change":"2023-01-25T08:56:44.571864+0000","last_active":"2023-01-25T08:56:47.251165+0000","last_peered":"2023-01-25T08:56:47.251165+0000","last_clean":"2023-01-25T08:56:47.251165+0000","last_became_active":"2023-01-25T08:56:44.570852+0000","last_became_peered":"2023-01-25T08:56:44.570852+0000","last_unstale":"2023-01-25T08:56:47.251165+0000","last_undegraded":"2023-01-25T08:56:47.251165+0000","last_fullsized":"2023-01-25T08:56:47.251165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_clean_scrub_stamp":"2023-01-25T08:56:42.760430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:47.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827790+0000","last_change":"2023-01-25T08:56:45.824516+0000","last_active":"2023-01-25T08:56:46.827790+0000","last_peered":"2023-01-25T08:56:46.827790+0000","last_clean":"2023-01-25T08:56:46.827790+0000","last_became_active":"2023-01-25T08:56:45.824228+0000","last_became_peered":"2023-01-25T08:56:45.824228+0000","last_unstale":"2023-01-25T08:56:46.827790+0000","last_undegraded":"2023-01-25T08:56:46.827790+0000","last_fullsized":"2023-01-25T08:56:46.827790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:51.431839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.826288+0000","last_change":"2023-01-25T08:56:45.823862+0000","last_active":"2023-01-25T08:56:46.826288+0000","last_peered":"2023-01-25T08:56:46.826288+0000","last_clean":"2023-01-25T08:56:46.826288+0000","last_became_active":"2023-01-25T08:56:45.823566+0000","last_became_peered":"2023-01-25T08:56:45.823566+0000","last_unstale":"2023-01-25T08:56:46.826288+0000","last_undegraded":"2023-01-25T08:56:46.826288+0000","last_fullsized":"2023-01-25T08:56:46.826288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:01.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827780+0000","last_change":"2023-01-25T08:56:45.824420+0000","last_active":"2023-01-25T08:56:46.827780+0000","last_peered":"2023-01-25T08:56:46.827780+0000","last_clean":"2023-01-25T08:56:46.827780+0000","last_became_active":"2023-01-25T08:56:45.824209+0000","last_became_peered":"2023-01-25T08:56:45.824209+0000","last_unstale":"2023-01-25T08:56:46.827780+0000","last_undegraded":"2023-01-25T08:56:46.827780+0000","last_fullsized":"2023-01-25T08:56:46.827780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:09.529598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.219870+0000","last_change":"2023-01-25T08:56:45.823947+0000","last_active":"2023-01-25T08:56:47.219870+0000","last_peered":"2023-01-25T08:56:47.219870+0000","last_clean":"2023-01-25T08:56:47.219870+0000","last_became_active":"2023-01-25T08:56:45.823652+0000","last_became_peered":"2023-01-25T08:56:45.823652+0000","last_unstale":"2023-01-25T08:56:47.219870+0000","last_undegraded":"2023-01-25T08:56:47.219870+0000","last_fullsized":"2023-01-25T08:56:47.219870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:53.603934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":43720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.8319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":43195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5552,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366288,"statfs":{"total":96636764160,"available":96631078912,"internally_reserved":0,"allocated":143360,"data_stored":567072,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":44309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":42457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":151552,"data_stored":571700,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T08:56:52.645 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T08:56:52.974 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-01-25T08:56:52.977 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-01-25T08:56:52.988 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2023-01-25T08:56:52.557208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41472,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":40320,"kb_avail":754933248,"statfs":{"total":773094113280,"available":773051645952,"internally_reserved":0,"allocated":1085440,"data_stored":1410779,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41287680},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000627"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827927+0000","last_change":"2023-01-25T08:56:46.542968+0000","last_active":"2023-01-25T08:56:46.827927+0000","last_peered":"2023-01-25T08:56:46.827927+0000","last_clean":"2023-01-25T08:56:46.827927+0000","last_became_active":"2023-01-25T08:56:46.542722+0000","last_became_peered":"2023-01-25T08:56:46.542722+0000","last_unstale":"2023-01-25T08:56:46.827927+0000","last_undegraded":"2023-01-25T08:56:46.827927+0000","last_fullsized":"2023-01-25T08:56:46.827927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:57.589071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.829553+0000","last_change":"2023-01-25T08:56:46.540445+0000","last_active":"2023-01-25T08:56:46.829553+0000","last_peered":"2023-01-25T08:56:46.829553+0000","last_clean":"2023-01-25T08:56:46.829553+0000","last_became_active":"2023-01-25T08:56:46.540172+0000","last_became_peered":"2023-01-25T08:56:46.540172+0000","last_unstale":"2023-01-25T08:56:46.829553+0000","last_undegraded":"2023-01-25T08:56:46.829553+0000","last_fullsized":"2023-01-25T08:56:46.829553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.284924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827870+0000","last_change":"2023-01-25T08:56:45.823843+0000","last_active":"2023-01-25T08:56:46.827870+0000","last_peered":"2023-01-25T08:56:46.827870+0000","last_clean":"2023-01-25T08:56:46.827870+0000","last_became_active":"2023-01-25T08:56:45.823567+0000","last_became_peered":"2023-01-25T08:56:45.823567+0000","last_unstale":"2023-01-25T08:56:46.827870+0000","last_undegraded":"2023-01-25T08:56:46.827870+0000","last_fullsized":"2023-01-25T08:56:46.827870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:02.016887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827781+0000","last_change":"2023-01-25T08:56:45.827804+0000","last_active":"2023-01-25T08:56:46.827781+0000","last_peered":"2023-01-25T08:56:46.827781+0000","last_clean":"2023-01-25T08:56:46.827781+0000","last_became_active":"2023-01-25T08:56:45.827629+0000","last_became_peered":"2023-01-25T08:56:45.827629+0000","last_unstale":"2023-01-25T08:56:46.827781+0000","last_undegraded":"2023-01-25T08:56:46.827781+0000","last_fullsized":"2023-01-25T08:56:46.827781+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:46.572338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.251165+0000","last_change":"2023-01-25T08:56:44.571864+0000","last_active":"2023-01-25T08:56:47.251165+0000","last_peered":"2023-01-25T08:56:47.251165+0000","last_clean":"2023-01-25T08:56:47.251165+0000","last_became_active":"2023-01-25T08:56:44.570852+0000","last_became_peered":"2023-01-25T08:56:44.570852+0000","last_unstale":"2023-01-25T08:56:47.251165+0000","last_undegraded":"2023-01-25T08:56:47.251165+0000","last_fullsized":"2023-01-25T08:56:47.251165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:42.760430+0000","last_clean_scrub_stamp":"2023-01-25T08:56:42.760430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:47.470130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827790+0000","last_change":"2023-01-25T08:56:45.824516+0000","last_active":"2023-01-25T08:56:46.827790+0000","last_peered":"2023-01-25T08:56:46.827790+0000","last_clean":"2023-01-25T08:56:46.827790+0000","last_became_active":"2023-01-25T08:56:45.824228+0000","last_became_peered":"2023-01-25T08:56:45.824228+0000","last_unstale":"2023-01-25T08:56:46.827790+0000","last_undegraded":"2023-01-25T08:56:46.827790+0000","last_fullsized":"2023-01-25T08:56:46.827790+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:51.431839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.826288+0000","last_change":"2023-01-25T08:56:45.823862+0000","last_active":"2023-01-25T08:56:46.826288+0000","last_peered":"2023-01-25T08:56:46.826288+0000","last_clean":"2023-01-25T08:56:46.826288+0000","last_became_active":"2023-01-25T08:56:45.823566+0000","last_became_peered":"2023-01-25T08:56:45.823566+0000","last_unstale":"2023-01-25T08:56:46.826288+0000","last_undegraded":"2023-01-25T08:56:46.826288+0000","last_fullsized":"2023-01-25T08:56:46.826288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:01.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T08:56:46.827780+0000","last_change":"2023-01-25T08:56:45.824420+0000","last_active":"2023-01-25T08:56:46.827780+0000","last_peered":"2023-01-25T08:56:46.827780+0000","last_clean":"2023-01-25T08:56:46.827780+0000","last_became_active":"2023-01-25T08:56:45.824209+0000","last_became_peered":"2023-01-25T08:56:45.824209+0000","last_unstale":"2023-01-25T08:56:46.827780+0000","last_undegraded":"2023-01-25T08:56:46.827780+0000","last_fullsized":"2023-01-25T08:56:46.827780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:58:09.529598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T08:56:47.219870+0000","last_change":"2023-01-25T08:56:45.823947+0000","last_active":"2023-01-25T08:56:47.219870+0000","last_peered":"2023-01-25T08:56:47.219870+0000","last_clean":"2023-01-25T08:56:47.219870+0000","last_became_active":"2023-01-25T08:56:45.823652+0000","last_became_peered":"2023-01-25T08:56:45.823652+0000","last_unstale":"2023-01-25T08:56:47.219870+0000","last_undegraded":"2023-01-25T08:56:47.219870+0000","last_fullsized":"2023-01-25T08:56:47.219870+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T08:56:44.775472+0000","last_clean_scrub_stamp":"2023-01-25T08:56:44.775472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T08:57:53.603934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9946,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5036,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366804,"statfs":{"total":96636764160,"available":96631607296,"internally_reserved":0,"allocated":102400,"data_stored":43720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6120000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.8319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":43195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5552,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366288,"statfs":{"total":96636764160,"available":96631078912,"internally_reserved":0,"allocated":143360,"data_stored":567072,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5076,"kb_used_data":140,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366764,"statfs":{"total":96636764160,"available":96631566336,"internally_reserved":0,"allocated":143360,"data_stored":44309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":42457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5084,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366756,"statfs":{"total":96636764160,"available":96631558144,"internally_reserved":0,"allocated":151552,"data_stored":49163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5532,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366308,"statfs":{"total":96636764160,"available":96631099392,"internally_reserved":0,"allocated":151552,"data_stored":571700,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4973,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T08:56:52.989 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T08:56:52.990 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-25T08:56:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 19 2023-01-25T08:56:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:fsid a4dc1c97-bb3d-4eba-b23f-bf25be3327d8 2023-01-25T08:56:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:created 2023-01-25T08:56:32.950433+0000 2023-01-25T08:56:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:modified 2023-01-25T08:56:51.897935+0000 2023-01-25T08:56:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-25T08:56:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:crush_version 5 2023-01-25T08:56:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:full_ratio 0.95 2023-01-25T08:56:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:backfillfull_ratio 0.9 2023-01-25T08:56:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:nearfull_ratio 0.85 2023-01-25T08:56:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:require_min_compat_client luminous 2023-01-25T08:56:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:min_compat_client jewel 2023-01-25T08:56:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:require_osd_release reef 2023-01-25T08:56:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:stretch_mode_enabled false 2023-01-25T08:56:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:pool 1 '.mgr' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-25T08:56:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:pool 2 'rbd' replicated size 2 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 application rbd 2023-01-25T08:56:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:max_osd 8 2023-01-25T08:56:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6812/106336 v1:172.21.15.6:6813/106336 exists,up d5589a5c-652d-41c4-9b5f-4440f7b52f45 2023-01-25T08:56:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6808/106335 v1:172.21.15.6:6809/106335 exists,up e5424e33-19e7-42c2-b7e7-de52c50d8aa7 2023-01-25T08:56:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6800/106333 v1:172.21.15.6:6801/106333 exists,up 29ac07f9-1427-47d4-af21-a70bcffb898e 2023-01-25T08:56:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6804/106334 v1:172.21.15.6:6805/106334 exists,up 35c680e5-76a0-4487-925b-5a02820a0c59 2023-01-25T08:56:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6808/106245 v1:172.21.15.8:6809/106245 exists,up b99283cb-7d71-47a4-a5f9-1e576f00d81b 2023-01-25T08:56:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6800/106235 v1:172.21.15.8:6801/106235 exists,up 737c7c42-b0c0-4cc7-9797-43b6ae7afb8c 2023-01-25T08:56:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6812/106251 v1:172.21.15.8:6813/106251 exists,up b8c3c2d2-84cb-465e-b559-94e344ead33b 2023-01-25T08:56:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6804/106236 v1:172.21.15.8:6805/106236 exists,up 5addbd03-a232-4959-bf02-167a968e5f77 2023-01-25T08:56:53.328 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6812/106336 v1:172.21.15.6:6813/106336 exists,up d5589a5c-652d-41c4-9b5f-4440f7b52f45', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6808/106335 v1:172.21.15.6:6809/106335 exists,up e5424e33-19e7-42c2-b7e7-de52c50d8aa7', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6800/106333 v1:172.21.15.6:6801/106333 exists,up 29ac07f9-1427-47d4-af21-a70bcffb898e', 'osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.6:6804/106334 v1:172.21.15.6:6805/106334 exists,up 35c680e5-76a0-4487-925b-5a02820a0c59', 'osd.4 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6808/106245 v1:172.21.15.8:6809/106245 exists,up b99283cb-7d71-47a4-a5f9-1e576f00d81b', 'osd.5 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6800/106235 v1:172.21.15.8:6801/106235 exists,up 737c7c42-b0c0-4cc7-9797-43b6ae7afb8c', 'osd.6 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6812/106251 v1:172.21.15.8:6813/106251 exists,up b8c3c2d2-84cb-465e-b559-94e344ead33b', 'osd.7 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) v1:172.21.15.8:6804/106236 v1:172.21.15.8:6805/106236 exists,up 5addbd03-a232-4959-bf02-167a968e5f77'] 2023-01-25T08:56:53.329 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-25T08:56:53.456 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-01-25T08:56:53.456 INFO:teuthology.orchestra.run.smithi006.stdout: "name": "mon.a", 2023-01-25T08:56:53.457 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster": "ceph", 2023-01-25T08:56:53.457 INFO:teuthology.orchestra.run.smithi006.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-25T08:56:53.458 INFO:teuthology.orchestra.run.smithi006.stdout: "admin_socket_mode": "", 2023-01-25T08:56:53.458 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-25T08:56:53.458 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_client_required": "cephx, none", 2023-01-25T08:56:53.459 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_cluster_required": "cephx", 2023-01-25T08:56:53.459 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_debug": "true", 2023-01-25T08:56:53.459 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-25T08:56:53.460 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-25T08:56:53.460 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_service_required": "cephx", 2023-01-25T08:56:53.460 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-25T08:56:53.461 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_supported": "cephx", 2023-01-25T08:56:53.461 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_aio": "true", 2023-01-25T08:56:53.461 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-25T08:56:53.461 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_aio_poll_ms": "250", 2023-01-25T08:56:53.461 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_aio_reap_max": "16", 2023-01-25T08:56:53.462 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_async_discard": "false", 2023-01-25T08:56:53.462 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_block_size": "4096", 2023-01-25T08:56:53.462 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_debug_aio": "false", 2023-01-25T08:56:53.462 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-25T08:56:53.462 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-25T08:56:53.463 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_debug_inflight_ios": "false", 2023-01-25T08:56:53.463 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_enable_discard": "false", 2023-01-25T08:56:53.463 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_flock_retry": "3", 2023-01-25T08:56:53.463 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-25T08:56:53.464 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_inject_crash": "0", 2023-01-25T08:56:53.464 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-25T08:56:53.464 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_ioring": "false", 2023-01-25T08:56:53.464 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_ioring_hipri": "false", 2023-01-25T08:56:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-25T08:56:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-25T08:56:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-25T08:56:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-25T08:56:53.465 INFO:teuthology.orchestra.run.smithi006.stdout: "bdev_type": "", 2023-01-25T08:56:53.466 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_alloc_size": "1048576", 2023-01-25T08:56:53.466 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_allocator": "hybrid", 2023-01-25T08:56:53.466 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_buffered_io": "true", 2023-01-25T08:56:53.466 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_check_for_zeros": "false", 2023-01-25T08:56:53.466 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-25T08:56:53.467 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-25T08:56:53.467 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_compact_log_sync": "false", 2023-01-25T08:56:53.467 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-25T08:56:53.467 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-25T08:56:53.467 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-25T08:56:53.468 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_max_log_runway": "4194304", 2023-01-25T08:56:53.468 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_max_prefetch": "1048576", 2023-01-25T08:56:53.468 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_min_flush_size": "524288", 2023-01-25T08:56:53.468 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_min_log_runway": "1048576", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_replay_recovery": "false", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluefs_sync_write": "false", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-25T08:56:53.469 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-25T08:56:53.470 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-25T08:56:53.470 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_allocation_from_file": "true", 2023-01-25T08:56:53.470 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_allocator": "hybrid", 2023-01-25T08:56:53.470 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-25T08:56:53.471 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-25T08:56:53.471 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-25T08:56:53.471 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-25T08:56:53.471 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-25T08:56:53.471 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-25T08:56:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-25T08:56:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_create": "true", 2023-01-25T08:56:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_db_create": "false", 2023-01-25T08:56:53.472 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_db_path": "", 2023-01-25T08:56:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_db_size": "0", 2023-01-25T08:56:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_path": "", 2023-01-25T08:56:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_preallocate_file": "false", 2023-01-25T08:56:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_size": "107374182400", 2023-01-25T08:56:53.473 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_wal_create": "false", 2023-01-25T08:56:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_wal_path": "", 2023-01-25T08:56:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_block_wal_size": "100663296", 2023-01-25T08:56:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bluefs": "true", 2023-01-25T08:56:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-25T08:56:53.474 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-25T08:56:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-25T08:56:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-25T08:56:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T08:56:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T08:56:53.475 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-25T08:56:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T08:56:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_autotune": "true", 2023-01-25T08:56:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-25T08:56:53.476 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-25T08:56:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-25T08:56:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-25T08:56:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_size": "0", 2023-01-25T08:56:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-25T08:56:53.477 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-25T08:56:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-25T08:56:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-25T08:56:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cache_type": "2q", 2023-01-25T08:56:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-25T08:56:53.478 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_clone_cow": "true", 2023-01-25T08:56:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-25T08:56:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-25T08:56:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-25T08:56:53.479 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-25T08:56:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-25T08:56:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-25T08:56:53.480 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-25T08:56:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_mode": "none", 2023-01-25T08:56:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-25T08:56:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_csum_type": "crc32c", 2023-01-25T08:56:53.481 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-25T08:56:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_freelist": "false", 2023-01-25T08:56:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-25T08:56:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-01-25T08:56:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-25T08:56:53.482 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-25T08:56:53.483 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-25T08:56:53.483 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-25T08:56:53.483 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-25T08:56:53.483 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-25T08:56:53.483 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-25T08:56:53.484 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-25T08:56:53.484 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-25T08:56:53.484 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-25T08:56:53.484 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-25T08:56:53.484 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-25T08:56:53.485 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_small_allocations": "0", 2023-01-25T08:56:53.485 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-25T08:56:53.485 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_default_buffered_read": "true", 2023-01-25T08:56:53.486 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_default_buffered_write": "false", 2023-01-25T08:56:53.486 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-25T08:56:53.486 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-25T08:56:53.486 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-25T08:56:53.486 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-25T08:56:53.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-25T08:56:53.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-25T08:56:53.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-25T08:56:53.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-25T08:56:53.487 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fail_eio": "false", 2023-01-25T08:56:53.488 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-25T08:56:53.488 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-25T08:56:53.488 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-25T08:56:53.489 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-25T08:56:53.489 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-25T08:56:53.489 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-25T08:56:53.489 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_mount": "false", 2023-01-25T08:56:53.489 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-25T08:56:53.490 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_umount": "false", 2023-01-25T08:56:53.490 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-25T08:56:53.490 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-25T08:56:53.490 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-25T08:56:53.490 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-25T08:56:53.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-25T08:56:53.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-25T08:56:53.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-25T08:56:53.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-25T08:56:53.491 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_ignore_data_csum": "false", 2023-01-25T08:56:53.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-25T08:56:53.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-25T08:56:53.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-25T08:56:53.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-25T08:56:53.492 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_log_op_age": "5.000000", 2023-01-25T08:56:53.493 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_alloc_size": "0", 2023-01-25T08:56:53.493 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_blob_size": "0", 2023-01-25T08:56:53.493 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-25T08:56:53.493 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-25T08:56:53.493 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-25T08:56:53.494 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_max_deferred_txc": "32", 2023-01-25T08:56:53.494 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_min_alloc_size": "0", 2023-01-25T08:56:53.494 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-25T08:56:53.495 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-25T08:56:53.495 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_nid_prealloc": "1024", 2023-01-25T08:56:53.495 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-25T08:56:53.495 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-25T08:56:53.496 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-25T08:56:53.496 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-25T08:56:53.497 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_retry_disk_reads": "3", 2023-01-25T08:56:53.497 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_rocksdb_cf": "true", 2023-01-25T08:56:53.497 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-25T08:56:53.497 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-01-25T08:56:53.497 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_spdk_mem": "512", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-25T08:56:53.498 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-25T08:56:53.499 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-25T08:56:53.500 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_tracing": "false", 2023-01-25T08:56:53.500 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-25T08:56:53.500 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-25T08:56:53.500 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-25T08:56:53.500 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-25T08:56:53.501 INFO:teuthology.orchestra.run.smithi006.stdout: "bluestore_zero_block_detection": "false", 2023-01-25T08:56:53.502 INFO:teuthology.orchestra.run.smithi006.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-25T08:56:53.502 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-25T08:56:53.502 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-25T08:56:53.502 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-25T08:56:53.502 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-25T08:56:53.503 INFO:teuthology.orchestra.run.smithi006.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-25T08:56:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-25T08:56:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-25T08:56:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_cluster_require_signatures": "false", 2023-01-25T08:56:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_cluster_require_version": "2", 2023-01-25T08:56:53.504 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_require_signatures": "false", 2023-01-25T08:56:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_require_version": "2", 2023-01-25T08:56:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_service_require_signatures": "false", 2023-01-25T08:56:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_service_require_version": "2", 2023-01-25T08:56:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "cephx_sign_messages": "true", 2023-01-25T08:56:53.505 INFO:teuthology.orchestra.run.smithi006.stdout: "chdir": "", 2023-01-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi006.stdout: "client_acl_type": "", 2023-01-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi006.stdout: "client_asio_thread_count": "2", 2023-01-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi006.stdout: "client_cache_mid": "0.750000", 2023-01-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi006.stdout: "client_cache_size": "16384", 2023-01-25T08:56:53.506 INFO:teuthology.orchestra.run.smithi006.stdout: "client_caps_release_delay": "5", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_check_pool_perm": "true", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_debug_force_sync_read": "false", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_debug_getattr_caps": "false", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_debug_inject_tick_delay": "0", 2023-01-25T08:56:53.507 INFO:teuthology.orchestra.run.smithi006.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-25T08:56:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: "client_die_on_failed_remount": "false", 2023-01-25T08:56:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: "client_dirsize_rbytes": "true", 2023-01-25T08:56:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: "client_force_lazyio": "false", 2023-01-25T08:56:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: "client_fs": "", 2023-01-25T08:56:53.508 INFO:teuthology.orchestra.run.smithi006.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-25T08:56:53.509 INFO:teuthology.orchestra.run.smithi006.stdout: "client_inject_release_failure": "false", 2023-01-25T08:56:53.509 INFO:teuthology.orchestra.run.smithi006.stdout: "client_max_inline_size": "4096", 2023-01-25T08:56:53.509 INFO:teuthology.orchestra.run.smithi006.stdout: "client_mds_namespace": "", 2023-01-25T08:56:53.509 INFO:teuthology.orchestra.run.smithi006.stdout: "client_metadata": "", 2023-01-25T08:56:53.509 INFO:teuthology.orchestra.run.smithi006.stdout: "client_mount_gid": "-1", 2023-01-25T08:56:53.510 INFO:teuthology.orchestra.run.smithi006.stdout: "client_mount_timeout": "300", 2023-01-25T08:56:53.510 INFO:teuthology.orchestra.run.smithi006.stdout: "client_mount_uid": "-1", 2023-01-25T08:56:53.510 INFO:teuthology.orchestra.run.smithi006.stdout: "client_mountpoint": "/", 2023-01-25T08:56:53.510 INFO:teuthology.orchestra.run.smithi006.stdout: "client_notify_timeout": "10", 2023-01-25T08:56:53.510 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc": "true", 2023-01-25T08:56:53.511 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc_max_dirty": "104857600", 2023-01-25T08:56:53.511 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-25T08:56:53.511 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc_max_objects": "1000", 2023-01-25T08:56:53.511 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc_size": "209715200", 2023-01-25T08:56:53.511 INFO:teuthology.orchestra.run.smithi006.stdout: "client_oc_target_dirty": "8388608", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_permissions": "true", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_quota": "true", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_quota_df": "true", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_readahead_max_bytes": "0", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_readahead_max_periods": "4", 2023-01-25T08:56:53.512 INFO:teuthology.orchestra.run.smithi006.stdout: "client_readahead_min": "131072", 2023-01-25T08:56:53.513 INFO:teuthology.orchestra.run.smithi006.stdout: "client_reconnect_stale": "false", 2023-01-25T08:56:53.513 INFO:teuthology.orchestra.run.smithi006.stdout: "client_shutdown_timeout": "30", 2023-01-25T08:56:53.513 INFO:teuthology.orchestra.run.smithi006.stdout: "client_snapdir": ".snap", 2023-01-25T08:56:53.513 INFO:teuthology.orchestra.run.smithi006.stdout: "client_tick_interval": "1", 2023-01-25T08:56:53.513 INFO:teuthology.orchestra.run.smithi006.stdout: "client_trace": "", 2023-01-25T08:56:53.514 INFO:teuthology.orchestra.run.smithi006.stdout: "client_try_dentry_invalidate": "false", 2023-01-25T08:56:53.514 INFO:teuthology.orchestra.run.smithi006.stdout: "client_use_faked_inos": "false", 2023-01-25T08:56:53.514 INFO:teuthology.orchestra.run.smithi006.stdout: "client_use_random_mds": "false", 2023-01-25T08:56:53.514 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_graylog": "false", 2023-01-25T08:56:53.514 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-25T08:56:53.515 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_graylog_port": "12201", 2023-01-25T08:56:53.515 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_monitors": "default=true", 2023-01-25T08:56:53.515 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_syslog": "false", 2023-01-25T08:56:53.515 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-25T08:56:53.515 INFO:teuthology.orchestra.run.smithi006.stdout: "clog_to_syslog_level": "info", 2023-01-25T08:56:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_addr": "-", 2023-01-25T08:56:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_network": "", 2023-01-25T08:56:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_network_interface": "", 2023-01-25T08:56:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: "compressor_zlib_isal": "false", 2023-01-25T08:56:53.516 INFO:teuthology.orchestra.run.smithi006.stdout: "compressor_zlib_level": "5", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "compressor_zlib_winsize": "-15", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "compressor_zstd_level": "1", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "crimson_alien_op_num_threads": "6", 2023-01-25T08:56:53.517 INFO:teuthology.orchestra.run.smithi006.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-25T08:56:53.518 INFO:teuthology.orchestra.run.smithi006.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-25T08:56:53.518 INFO:teuthology.orchestra.run.smithi006.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-25T08:56:53.518 INFO:teuthology.orchestra.run.smithi006.stdout: "crimson_seastar_smp": "1", 2023-01-25T08:56:53.518 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_location": "", 2023-01-25T08:56:53.518 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_location_hook": "", 2023-01-25T08:56:53.519 INFO:teuthology.orchestra.run.smithi006.stdout: "crush_location_hook_timeout": "10", 2023-01-25T08:56:53.519 INFO:teuthology.orchestra.run.smithi006.stdout: "daemonize": "false", 2023-01-25T08:56:53.519 INFO:teuthology.orchestra.run.smithi006.stdout: "daos_pool": "tank", 2023-01-25T08:56:53.519 INFO:teuthology.orchestra.run.smithi006.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-01-25T08:56:53.519 INFO:teuthology.orchestra.run.smithi006.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-01-25T08:56:53.520 INFO:teuthology.orchestra.run.smithi006.stdout: "dbstore_db_name_prefix": "dbstore", 2023-01-25T08:56:53.520 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_alienstore": "0/5", 2023-01-25T08:56:53.520 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_allow_any_pool_priority": "false", 2023-01-25T08:56:53.520 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_asok": "1/5", 2023-01-25T08:56:53.520 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_asok_assert_abort": "false", 2023-01-25T08:56:53.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_asserts_on_shutdown": "true", 2023-01-25T08:56:53.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_auth": "20/20", 2023-01-25T08:56:53.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_bdev": "1/3", 2023-01-25T08:56:53.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_bluefs": "1/5", 2023-01-25T08:56:53.521 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_bluestore": "1/5", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_buffer": "0/1", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_ceph_exporter": "1/5", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_cephfs_mirror": "0/5", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_cephsqlite": "0/5", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_client": "0/5", 2023-01-25T08:56:53.522 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_compressor": "1/5", 2023-01-25T08:56:53.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_context": "0/1", 2023-01-25T08:56:53.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_crush": "1/1", 2023-01-25T08:56:53.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_crypto": "1/5", 2023-01-25T08:56:53.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_cyanstore": "0/5", 2023-01-25T08:56:53.523 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_deliberately_leak_memory": "false", 2023-01-25T08:56:53.524 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_disable_randomized_ping": "false", 2023-01-25T08:56:53.524 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_dpdk": "1/5", 2023-01-25T08:56:53.524 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_eventtrace": "1/5", 2023-01-25T08:56:53.524 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_filer": "0/1", 2023-01-25T08:56:53.524 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_filestore": "1/3", 2023-01-25T08:56:53.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_finisher": "1/1", 2023-01-25T08:56:53.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_fuse": "1/5", 2023-01-25T08:56:53.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_heartbeat_testing_span": "0", 2023-01-25T08:56:53.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_heartbeatmap": "1/5", 2023-01-25T08:56:53.525 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_javaclient": "1/5", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_journal": "1/3", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_journaler": "0/5", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_kstore": "1/5", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_leveldb": "4/5", 2023-01-25T08:56:53.526 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_lockdep": "0/1", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mclock": "1/5", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds": "1/5", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds_balancer": "1/5", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds_locker": "1/5", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds_log": "1/5", 2023-01-25T08:56:53.527 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds_log_expire": "1/5", 2023-01-25T08:56:53.528 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mds_migrator": "1/5", 2023-01-25T08:56:53.528 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_memstore": "1/5", 2023-01-25T08:56:53.528 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mgr": "2/5", 2023-01-25T08:56:53.528 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mgrc": "1/5", 2023-01-25T08:56:53.528 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_mon": "20/20", 2023-01-25T08:56:53.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_monc": "0/10", 2023-01-25T08:56:53.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_ms": "1/1", 2023-01-25T08:56:53.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_none": "0/5", 2023-01-25T08:56:53.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_objclass": "0/5", 2023-01-25T08:56:53.529 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_objectcacher": "0/5", 2023-01-25T08:56:53.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_objecter": "0/1", 2023-01-25T08:56:53.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_optracker": "0/5", 2023-01-25T08:56:53.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_osd": "1/5", 2023-01-25T08:56:53.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_paxos": "20/20", 2023-01-25T08:56:53.530 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_perfcounter": "1/5", 2023-01-25T08:56:53.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_prioritycache": "1/5", 2023-01-25T08:56:53.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rados": "0/5", 2023-01-25T08:56:53.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rbd": "0/5", 2023-01-25T08:56:53.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rbd_mirror": "0/5", 2023-01-25T08:56:53.531 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rbd_pwl": "0/5", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rbd_replay": "0/5", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_refs": "0/0", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_reserver": "1/1", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw": "1/5", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw_access": "1/5", 2023-01-25T08:56:53.532 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw_datacache": "1/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw_dbstore": "1/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw_flight": "1/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rgw_sync": "1/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_rocksdb": "4/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore": "0/5", 2023-01-25T08:56:53.533 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_backref": "0/5", 2023-01-25T08:56:53.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_cache": "0/5", 2023-01-25T08:56:53.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_cleaner": "0/5", 2023-01-25T08:56:53.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_device": "0/5", 2023-01-25T08:56:53.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_epm": "0/5", 2023-01-25T08:56:53.534 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-01-25T08:56:53.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_journal": "0/5", 2023-01-25T08:56:53.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_lba": "0/5", 2023-01-25T08:56:53.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_odata": "0/5", 2023-01-25T08:56:53.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_omap": "0/5", 2023-01-25T08:56:53.535 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_onode": "0/5", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_t": "0/5", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_seastore_tm": "0/5", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_striper": "0/1", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_test": "0/5", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_throttle": "1/1", 2023-01-25T08:56:53.536 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_timer": "0/1", 2023-01-25T08:56:53.537 INFO:teuthology.orchestra.run.smithi006.stdout: "debug_tp": "0/5", 2023-01-25T08:56:53.537 INFO:teuthology.orchestra.run.smithi006.stdout: "device_failure_prediction_mode": "none", 2023-01-25T08:56:53.537 INFO:teuthology.orchestra.run.smithi006.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-25T08:56:53.537 INFO:teuthology.orchestra.run.smithi006.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-25T08:56:53.537 INFO:teuthology.orchestra.run.smithi006.stdout: "err_to_graylog": "false", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "err_to_journald": "false", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "err_to_stderr": "true", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "err_to_syslog": "false", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "event_tracing": "false", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_addr": "0.0.0.0", 2023-01-25T08:56:53.538 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_http_port": "9926", 2023-01-25T08:56:53.539 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_prio_limit": "5", 2023-01-25T08:56:53.539 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-25T08:56:53.539 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_sort_metrics": "true", 2023-01-25T08:56:53.539 INFO:teuthology.orchestra.run.smithi006.stdout: "exporter_stats_period": "5", 2023-01-25T08:56:53.539 INFO:teuthology.orchestra.run.smithi006.stdout: "fake_statfs_for_testing": "0", 2023-01-25T08:56:53.540 INFO:teuthology.orchestra.run.smithi006.stdout: "fatal_signal_handlers": "true", 2023-01-25T08:56:53.540 INFO:teuthology.orchestra.run.smithi006.stdout: "filer_max_purge_ops": "10", 2023-01-25T08:56:53.540 INFO:teuthology.orchestra.run.smithi006.stdout: "filer_max_truncate_ops": "128", 2023-01-25T08:56:53.540 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_apply_finisher_threads": "1", 2023-01-25T08:56:53.540 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_blackhole": "false", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_btrfs_clone_range": "true", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_btrfs_snap": "true", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_caller_concurrency": "10", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_collect_device_partition_information": "true", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_commit_timeout": "600.000000", 2023-01-25T08:56:53.541 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_debug_inject_read_err": "false", 2023-01-25T08:56:53.542 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_debug_omap_check": "false", 2023-01-25T08:56:53.542 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-25T08:56:53.542 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_debug_verify_split": "false", 2023-01-25T08:56:53.542 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_dump_file": "", 2023-01-25T08:56:53.542 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-25T08:56:53.543 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-25T08:56:53.543 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fadvise": "true", 2023-01-25T08:56:53.543 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fail_eio": "true", 2023-01-25T08:56:53.543 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fd_cache_shards": "16", 2023-01-25T08:56:53.543 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fd_cache_size": "128", 2023-01-25T08:56:53.544 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fiemap": "false", 2023-01-25T08:56:53.544 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fiemap_threshold": "4096", 2023-01-25T08:56:53.544 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-25T08:56:53.544 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-25T08:56:53.544 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_inject_stall": "0", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_journal_parallel": "false", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_journal_trailing": "false", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_journal_writeahead": "false", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_kill_at": "0", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-25T08:56:53.545 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-25T08:56:53.546 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-25T08:56:53.546 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-25T08:56:53.546 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-25T08:56:53.546 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattrs": "0", 2023-01-25T08:56:53.546 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_xattr_value_size": "0", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-25T08:56:53.547 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-25T08:56:53.548 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-25T08:56:53.548 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_merge_threshold": "-10", 2023-01-25T08:56:53.548 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-25T08:56:53.548 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_odsync_write": "false", 2023-01-25T08:56:53.548 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_omap_backend": "rocksdb", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_omap_backend_path": "", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_op_thread_timeout": "60", 2023-01-25T08:56:53.549 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_op_threads": "2", 2023-01-25T08:56:53.550 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_punch_hole": "false", 2023-01-25T08:56:53.550 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-25T08:56:53.550 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-25T08:56:53.550 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-25T08:56:53.550 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-25T08:56:53.551 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-25T08:56:53.551 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-25T08:56:53.551 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-25T08:56:53.551 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-25T08:56:53.551 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-25T08:56:53.552 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_queue_max_ops": "50", 2023-01-25T08:56:53.552 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-25T08:56:53.552 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_seek_data_hole": "false", 2023-01-25T08:56:53.552 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_sloppy_crc": "false", 2023-01-25T08:56:53.552 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-25T08:56:53.553 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_splice": "false", 2023-01-25T08:56:53.553 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_split_multiple": "2", 2023-01-25T08:56:53.553 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_split_rand_factor": "20", 2023-01-25T08:56:53.553 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_update_to": "1000", 2023-01-25T08:56:53.553 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-25T08:56:53.554 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-25T08:56:53.554 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-25T08:56:53.554 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-25T08:56:53.554 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-25T08:56:53.554 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_enable": "true", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-25T08:56:53.555 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-25T08:56:53.556 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-25T08:56:53.556 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_xfs_extsize": "false", 2023-01-25T08:56:53.556 INFO:teuthology.orchestra.run.smithi006.stdout: "filestore_zfs_snap": "false", 2023-01-25T08:56:53.556 INFO:teuthology.orchestra.run.smithi006.stdout: "fio_dir": "/tmp/fio", 2023-01-25T08:56:53.556 INFO:teuthology.orchestra.run.smithi006.stdout: "fsid": "a4dc1c97-bb3d-4eba-b23f-bf25be3327d8", 2023-01-25T08:56:53.557 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_allow_other": "true", 2023-01-25T08:56:53.557 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_atomic_o_trunc": "true", 2023-01-25T08:56:53.557 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_big_writes": "true", 2023-01-25T08:56:53.557 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_debug": "false", 2023-01-25T08:56:53.557 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_default_permissions": "false", 2023-01-25T08:56:53.558 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_disable_pagecache": "false", 2023-01-25T08:56:53.558 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_max_write": "0", 2023-01-25T08:56:53.558 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_multithreaded": "true", 2023-01-25T08:56:53.558 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_require_active_mds": "true", 2023-01-25T08:56:53.558 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_set_user_groups": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_splice_move": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_splice_read": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_splice_write": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "fuse_use_invalidate_cb": "true", 2023-01-25T08:56:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi006.stdout: "gss_target_name": "ceph", 2023-01-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi006.stdout: "heartbeat_file": "", 2023-01-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi006.stdout: "heartbeat_inject_failure": "0", 2023-01-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi006.stdout: "heartbeat_interval": "5", 2023-01-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi006.stdout: "host": "smithi006", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-25T08:56:53.563 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-25T08:56:53.563 INFO:teuthology.orchestra.run.smithi006.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-25T08:56:53.563 INFO:teuthology.orchestra.run.smithi006.stdout: "inject_early_sigterm": "false", 2023-01-25T08:56:53.563 INFO:teuthology.orchestra.run.smithi006.stdout: "jaeger_tracing_enable": "false", 2023-01-25T08:56:53.563 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_aio": "true", 2023-01-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_align_min_size": "65536", 2023-01-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_block_align": "true", 2023-01-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_block_size": "4096", 2023-01-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_dio": "true", 2023-01-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_discard": "false", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_force_aio": "false", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_ignore_corruption": "false", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_max_write_bytes": "10485760", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_max_write_entries": "100", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_replay_from": "0", 2023-01-25T08:56:53.565 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_write_header_frequency": "0", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journal_zero_on_create": "false", 2023-01-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi006.stdout: "journaler_prefetch_periods": "10", 2023-01-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi006.stdout: "journaler_prezero_periods": "5", 2023-01-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi006.stdout: "journaler_write_head_interval": "15", 2023-01-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "", 2023-01-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi006.stdout: "keyfile": "", 2023-01-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi006.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_backend": "rocksdb", 2023-01-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_default_stripe_size": "65536", 2023-01-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_fsck_on_mount": "false", 2023-01-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_max_bytes": "67108864", 2023-01-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_max_ops": "512", 2023-01-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_nid_prealloc": "1024", 2023-01-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_onode_map_size": "1024", 2023-01-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_sync_submit_transaction": "false", 2023-01-25T08:56:53.570 INFO:teuthology.orchestra.run.smithi006.stdout: "kstore_sync_transaction": "false", 2023-01-25T08:56:53.570 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_block_size": "65536", 2023-01-25T08:56:53.570 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_bloom_size": "0", 2023-01-25T08:56:53.570 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_cache_size": "536870912", 2023-01-25T08:56:53.570 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_compact_on_mount": "false", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_compression": "false", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_log": "", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_max_open_files": "0", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_paranoid": "false", 2023-01-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi006.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-25T08:56:53.572 INFO:teuthology.orchestra.run.smithi006.stdout: "librados_thread_count": "2", 2023-01-25T08:56:53.572 INFO:teuthology.orchestra.run.smithi006.stdout: "lockdep": "false", 2023-01-25T08:56:53.572 INFO:teuthology.orchestra.run.smithi006.stdout: "lockdep_force_backtrace": "false", 2023-01-25T08:56:53.572 INFO:teuthology.orchestra.run.smithi006.stdout: "log_coarse_timestamps": "true", 2023-01-25T08:56:53.572 INFO:teuthology.orchestra.run.smithi006.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-25T08:56:53.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_flush_on_exit": "false", 2023-01-25T08:56:53.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_graylog_host": "127.0.0.1", 2023-01-25T08:56:53.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_graylog_port": "12201", 2023-01-25T08:56:53.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_max_new": "1000", 2023-01-25T08:56:53.573 INFO:teuthology.orchestra.run.smithi006.stdout: "log_max_recent": "10000", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_stderr_prefix": "", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_stop_at_utilization": "0.970000", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_file": "true", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_graylog": "false", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_journald": "false", 2023-01-25T08:56:53.574 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_stderr": "false", 2023-01-25T08:56:53.575 INFO:teuthology.orchestra.run.smithi006.stdout: "log_to_syslog": "false", 2023-01-25T08:56:53.575 INFO:teuthology.orchestra.run.smithi006.stdout: "max_rotating_auth_attempts": "10", 2023-01-25T08:56:53.575 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_action_on_write_error": "1", 2023-01-25T08:56:53.575 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_alternate_name_max": "8192", 2023-01-25T08:56:53.575 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_asio_thread_count": "2", 2023-01-25T08:56:53.576 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_export_pin": "true", 2023-01-25T08:56:53.576 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_fragment_dirs": "true", 2023-01-25T08:56:53.576 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-25T08:56:53.576 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_fragment_interval": "5", 2023-01-25T08:56:53.576 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_interval": "10", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_max": "-1", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_max_until": "-1", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_merge_size": "50", 2023-01-25T08:56:53.577 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_midchunk": "0.300000", 2023-01-25T08:56:53.578 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-25T08:56:53.578 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_min_start": "0.200000", 2023-01-25T08:56:53.578 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_minchunk": "0.001000", 2023-01-25T08:56:53.578 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_mode": "0", 2023-01-25T08:56:53.578 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_need_max": "1.200000", 2023-01-25T08:56:53.579 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_need_min": "0.800000", 2023-01-25T08:56:53.579 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-25T08:56:53.579 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-25T08:56:53.579 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_split_bits": "3", 2023-01-25T08:56:53.579 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-25T08:56:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_split_size": "10000", 2023-01-25T08:56:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-25T08:56:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_target_decay": "10.000000", 2023-01-25T08:56:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-25T08:56:53.580 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_beacon_grace": "15.000000", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_beacon_interval": "4.000000", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_mid": "0.700000", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_release_free_interval": "10", 2023-01-25T08:56:53.581 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_reservation": "0.050000", 2023-01-25T08:56:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-25T08:56:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_trim_interval": "1", 2023-01-25T08:56:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cache_trim_threshold": "262144", 2023-01-25T08:56:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-25T08:56:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-25T08:56:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-25T08:56:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_client_prealloc_inos": "1000", 2023-01-25T08:56:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-25T08:56:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_connect_bootstrapping": "false", 2023-01-25T08:56:53.583 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_damage_table_max_entries": "10000", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_debug_auth_pins": "false", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_debug_frag": "false", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_debug_scatterstat": "false", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_debug_subtrees": "false", 2023-01-25T08:56:53.584 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_decay_halflife": "5.000000", 2023-01-25T08:56:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_default_dir_hash": "2", 2023-01-25T08:56:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_defer_session_stale": "true", 2023-01-25T08:56:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_deny_all_reconnect": "false", 2023-01-25T08:56:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dir_keys_per_op": "16384", 2023-01-25T08:56:53.585 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dir_max_commit_size": "10", 2023-01-25T08:56:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dir_max_entries": "0", 2023-01-25T08:56:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dir_prefetch": "true", 2023-01-25T08:56:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-25T08:56:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-25T08:56:53.586 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dump_cache_on_map": "false", 2023-01-25T08:56:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-25T08:56:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-25T08:56:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_early_reply": "true", 2023-01-25T08:56:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_enable_op_tracker": "true", 2023-01-25T08:56:53.587 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_enforce_unique_name": "true", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_export_ephemeral_random": "true", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-01-25T08:56:53.588 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-25T08:56:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-25T08:56:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-01-25T08:56:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-25T08:56:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-25T08:56:53.589 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_health_summarize_threshold": "10", 2023-01-25T08:56:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-25T08:56:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-25T08:56:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_inject_health_dummy": "false", 2023-01-25T08:56:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_inject_migrator_session_race": "false", 2023-01-25T08:56:53.590 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_join_fs": "", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_journal_format": "1", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_create_at": "0", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_export_at": "0", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_import_at": "0", 2023-01-25T08:56:53.591 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_journal_at": "0", 2023-01-25T08:56:53.592 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_journal_expire_at": "0", 2023-01-25T08:56:53.592 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_journal_replay_at": "0", 2023-01-25T08:56:53.592 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_link_at": "0", 2023-01-25T08:56:53.592 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_mdstable_at": "0", 2023-01-25T08:56:53.592 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_openc_at": "0", 2023-01-25T08:56:53.593 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_kill_rename_at": "0", 2023-01-25T08:56:53.593 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_events_per_segment": "1024", 2023-01-25T08:56:53.593 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_max_events": "-1", 2023-01-25T08:56:53.593 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_max_segments": "128", 2023-01-25T08:56:53.593 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_pause": "false", 2023-01-25T08:56:53.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_segment_size": "0", 2023-01-25T08:56:53.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-25T08:56:53.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_log_warn_factor": "2.000000", 2023-01-25T08:56:53.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_caps_per_client": "1048576", 2023-01-25T08:56:53.594 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_completed_flushes": "100000", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_completed_requests": "100000", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_export_size": "20971520", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_file_recover": "32", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_purge_files": "64", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_purge_ops": "8192", 2023-01-25T08:56:53.595 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-25T08:56:53.596 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-25T08:56:53.596 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-25T08:56:53.596 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_snaps_per_dir": "100", 2023-01-25T08:56:53.596 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-25T08:56:53.596 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_metrics_update_interval": "2", 2023-01-25T08:56:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_min_caps_per_client": "100", 2023-01-25T08:56:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_min_caps_working_set": "10000", 2023-01-25T08:56:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-25T08:56:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_numa_node": "-1", 2023-01-25T08:56:53.597 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_op_complaint_time": "30.000000", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_op_history_duration": "600", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_op_history_size": "20", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_op_log_threshold": "5", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_ping_grace": "15", 2023-01-25T08:56:53.598 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_ping_interval": "5", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_max_caps": "30000", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-25T08:56:53.599 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-25T08:56:53.600 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_recall_warning_threshold": "262144", 2023-01-25T08:56:53.600 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-25T08:56:53.600 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_replay_interval": "1.000000", 2023-01-25T08:56:53.600 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-25T08:56:53.600 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-25T08:56:53.601 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_root_ino_gid": "0", 2023-01-25T08:56:53.601 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_root_ino_uid": "0", 2023-01-25T08:56:53.601 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-25T08:56:53.601 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-25T08:56:53.601 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-25T08:56:53.602 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-25T08:56:53.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_shutdown_check": "0", 2023-01-25T08:56:53.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_skip_ino": "0", 2023-01-25T08:56:53.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-25T08:56:53.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_snap_max_uid": "4294967294", 2023-01-25T08:56:53.603 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_snap_min_uid": "0", 2023-01-25T08:56:53.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_snap_rstat": "false", 2023-01-25T08:56:53.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_standby_replay_damaged": "false", 2023-01-25T08:56:53.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_symlink_recovery": "true", 2023-01-25T08:56:53.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-25T08:56:53.604 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_thrash_exports": "0", 2023-01-25T08:56:53.605 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_thrash_fragments": "0", 2023-01-25T08:56:53.605 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_tick_interval": "5.000000", 2023-01-25T08:56:53.605 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_valgrind_exit": "false", 2023-01-25T08:56:53.605 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_verify_backtrace": "1", 2023-01-25T08:56:53.605 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_verify_scatter": "false", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_wipe_sessions": "false", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "mempool_debug": "false", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "memstore_device_bytes": "1073741824", 2023-01-25T08:56:53.606 INFO:teuthology.orchestra.run.smithi006.stdout: "memstore_page_set": "false", 2023-01-25T08:56:53.607 INFO:teuthology.orchestra.run.smithi006.stdout: "memstore_page_size": "65536", 2023-01-25T08:56:53.607 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_client_bytes": "134217728", 2023-01-25T08:56:53.607 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_client_messages": "512", 2023-01-25T08:56:53.607 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-25T08:56:53.607 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_disabled_modules": "", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_max_pg_num_change": "128", 2023-01-25T08:56:53.608 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_mds_bytes": "134217728", 2023-01-25T08:56:53.609 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_mds_messages": "128", 2023-01-25T08:56:53.609 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-25T08:56:53.609 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_mon_bytes": "134217728", 2023-01-25T08:56:53.609 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_mon_messages": "128", 2023-01-25T08:56:53.609 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_osd_bytes": "536870912", 2023-01-25T08:56:53.610 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_osd_messages": "8192", 2023-01-25T08:56:53.610 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_pool": "true", 2023-01-25T08:56:53.610 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-25T08:56:53.610 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_standby_modules": "true", 2023-01-25T08:56:53.610 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_stats_period": "5", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_stats_threshold": "5", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_tick_period": "2", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_allow_pool_delete": "true", 2023-01-25T08:56:53.611 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_allow_pool_size_one": "true", 2023-01-25T08:56:53.612 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_auth_validate_all_caps": "true", 2023-01-25T08:56:53.612 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-25T08:56:53.612 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-25T08:56:53.612 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_bytes": "104857600", 2023-01-25T08:56:53.612 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_directed_command_retry": "5", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_hunt_parallel": "3", 2023-01-25T08:56:53.613 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_log_interval": "1.000000", 2023-01-25T08:56:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-25T08:56:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_ping_interval": "10.000000", 2023-01-25T08:56:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-25T08:56:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-25T08:56:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-25T08:56:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-25T08:56:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_file_level": "debug", 2023-01-25T08:56:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_file": "true", 2023-01-25T08:56:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-25T08:56:53.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-25T08:56:53.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-25T08:56:53.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_journald": "false", 2023-01-25T08:56:53.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-25T08:56:53.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-25T08:56:53.616 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_compact_on_bootstrap": "false", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_compact_on_start": "false", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_compact_on_trim": "true", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-25T08:56:53.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-25T08:56:53.618 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-25T08:56:53.618 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_cpu_threads": "4", 2023-01-25T08:56:53.618 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_crush_min_required_version": "hammer", 2023-01-25T08:56:53.618 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_daemon_bytes": "419430400", 2023-01-25T08:56:53.618 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-25T08:56:53.619 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_data_avail_crit": "5", 2023-01-25T08:56:53.619 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_data_avail_warn": "5", 2023-01-25T08:56:53.619 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_data_size_warn": "16106127360", 2023-01-25T08:56:53.619 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-25T08:56:53.619 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-25T08:56:53.620 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_dump_json": "false", 2023-01-25T08:56:53.620 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-25T08:56:53.620 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_dump_transactions": "false", 2023-01-25T08:56:53.620 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_extra_checks": "false", 2023-01-25T08:56:53.620 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-25T08:56:53.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-25T08:56:53.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_no_require_quincy": "false", 2023-01-25T08:56:53.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_no_require_reef": "false", 2023-01-25T08:56:53.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-25T08:56:53.621 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_down_mkfs_grace": "120", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_election_default_strategy": "1", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_election_timeout": "5.000000", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-25T08:56:53.622 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_elector_ping_divisor": "2", 2023-01-25T08:56:53.623 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-25T08:56:53.623 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_enable_op_tracker": "true", 2023-01-25T08:56:53.623 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_fake_pool_delete": "false", 2023-01-25T08:56:53.623 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_force_quorum_join": "false", 2023-01-25T08:56:53.623 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_globalid_prealloc": "10000", 2023-01-25T08:56:53.624 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_detail_to_clog": "false", 2023-01-25T08:56:53.624 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_log_update_period": "5", 2023-01-25T08:56:53.624 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_max_detail": "50", 2023-01-25T08:56:53.624 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_to_clog": "true", 2023-01-25T08:56:53.624 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_to_clog_interval": "600", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_host": "[v1:172.21.15.6:6789],[v1:172.21.15.6:6790],[v1:172.21.15.8:6789]", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_host_override": "", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_initial_members": "", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-25T08:56:53.625 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-25T08:56:53.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-25T08:56:53.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-25T08:56:53.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-25T08:56:53.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_lease": "5.000000", 2023-01-25T08:56:53.626 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-25T08:56:53.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-25T08:56:53.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_log_full_interval": "50", 2023-01-25T08:56:53.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_log_max": "10000", 2023-01-25T08:56:53.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_log_max_summary": "50", 2023-01-25T08:56:53.627 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_log_epochs": "500", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_osd": "10000", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_pg_per_osd": "10000", 2023-01-25T08:56:53.628 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_pool_pg_num": "65536", 2023-01-25T08:56:53.629 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-25T08:56:53.629 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-25T08:56:53.629 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mds_force_trim_to": "0", 2023-01-25T08:56:53.629 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mds_skip_sanity": "false", 2023-01-25T08:56:53.629 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_memory_autotune": "true", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_memory_target": "2147483648", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_beacon_grace": "30", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_digest_period": "5", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_inactive_grace": "60", 2023-01-25T08:56:53.630 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-25T08:56:53.631 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-25T08:56:53.631 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_min_osdmap_epochs": "50", 2023-01-25T08:56:53.631 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_complaint_time": "30", 2023-01-25T08:56:53.631 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_history_duration": "600", 2023-01-25T08:56:53.631 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_history_size": "20", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_history_slow_op_size": "20", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_op_log_threshold": "5", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-25T08:56:53.632 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_auto_mark_in": "false", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_cache_size": "500", 2023-01-25T08:56:53.633 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-25T08:56:53.634 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-25T08:56:53.634 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-25T08:56:53.634 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_down_out_interval": "600", 2023-01-25T08:56:53.634 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-25T08:56:53.634 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-25T08:56:53.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_force_trim_to": "0", 2023-01-25T08:56:53.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-25T08:56:53.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-25T08:56:53.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-25T08:56:53.635 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-25T08:56:53.636 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-25T08:56:53.636 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-25T08:56:53.636 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-25T08:56:53.636 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-25T08:56:53.636 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_min_down_reporters": "2", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-25T08:56:53.637 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-25T08:56:53.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_report_timeout": "900", 2023-01-25T08:56:53.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-25T08:56:53.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-25T08:56:53.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-25T08:56:53.638 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osdmap_full_prune_interval": "2", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osdmap_full_prune_min": "15", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-25T08:56:53.639 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_stuck_threshold": "60", 2023-01-25T08:56:53.640 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-25T08:56:53.640 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-25T08:56:53.640 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-25T08:56:53.640 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-25T08:56:53.640 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_probe_timeout": "2.000000", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_reweight_max_change": "0.050000", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_reweight_max_osds": "4", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-25T08:56:53.641 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-25T08:56:53.642 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-25T08:56:53.642 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-25T08:56:53.642 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-25T08:56:53.642 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_scrub_interval": "300", 2023-01-25T08:56:53.642 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_scrub_max_keys": "100", 2023-01-25T08:56:53.643 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_scrub_timeout": "300", 2023-01-25T08:56:53.643 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_session_timeout": "300", 2023-01-25T08:56:53.643 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_smart_report_timeout": "5", 2023-01-25T08:56:53.643 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_stat_smooth_intervals": "6", 2023-01-25T08:56:53.643 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-25T08:56:53.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_stretch_pool_min_size": "2", 2023-01-25T08:56:53.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_stretch_pool_size": "4", 2023-01-25T08:56:53.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-25T08:56:53.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-25T08:56:53.644 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_debug": "false", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_provider_kill_at": "0", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_requester_kill_at": "0", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_sync_timeout": "60.000000", 2023-01-25T08:56:53.645 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_target_pg_per_osd": "100", 2023-01-25T08:56:53.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_tick_interval": "5", 2023-01-25T08:56:53.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_timecheck_interval": "300.000000", 2023-01-25T08:56:53.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-25T08:56:53.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_use_min_delay_socket": "false", 2023-01-25T08:56:53.646 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_older_version_delay": "604800", 2023-01-25T08:56:53.647 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-25T08:56:53.647 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-25T08:56:53.647 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-25T08:56:53.647 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-25T08:56:53.647 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_misplaced": "false", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_older_version": "true", 2023-01-25T08:56:53.648 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-25T08:56:53.649 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-25T08:56:53.649 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-25T08:56:53.649 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-25T08:56:53.649 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "monmap": "", 2023-01-25T08:56:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-01-25T08:56:53.651 INFO:teuthology.orchestra.run.smithi006.stdout: