2023-01-25T06:18:36.133 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T06:18:36.134 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917 --verbose --name yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917/orig.config.yaml 2023-01-25T06:18:36.183 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T06:18:36.255 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917 branch: wip-yuri2-testing-2023-01-23-0928 description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} email: yweinstein@radhat.com first_in_suite: false job_id: '7136917' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 - volumes: size: 50 - machine: ram: 15000 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri2-testing-2023-01-23-0928 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 2500 ms tcp read timeout: 5 ms type: async osd_async_recovery_min_cost: 1 osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore queue throttle high multiple: 2 filestore queue throttle max multiple: 10 journal throttle high multiple: 2 journal throttle max multiple: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 9 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - must scrub before tier agent can activate mon_bind_msgr2: false sha1: 61996d73ab738798e4ee0055c86457f5c459da4a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 61996d73ab738798e4ee0055c86457f5c459da4a selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-yuri2-testing-2023-01-23-0928 sha1: 61996d73ab738798e4ee0055c86457f5c459da4a owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 61996d73ab738798e4ee0055c86457f5c459da4a sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri2-testing-2023-01-23-0928 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_61996d73ab738798e4ee0055c86457f5c459da4a/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 61996d73ab738798e4ee0055c86457f5c459da4a targets: smithi062.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiU0p+LdQaepzlZFUYWJhRyMPNBzRtv7db6XpsJfTBfiPKc8GdAc7yW/vWWSfmFDMv7DoNgFnm+O5RULi5W4JL74+P5URruPK/lcqnKJvgvQnTnKIrW2pu+fLmiDGDd1Etabjpee/15u9/8UeGdASj0c2UIqV1+6PW0DcFMBOoq+Izlv4VYgRYHuDzvkqjiOKxDcB66yb9PBB24UyJUV2dDjApT/c0lJoNTeFxANb8yuRoSkXU34rBSGaW3oFtims/HdOoSjDojvmBnMMEMDgrPJIXtIowiaNPmKl3Rd5QbxMdW2TGxtry42DQK1AGHnmueJ6Wy9HJWTbgqiDlzRjWYeTDSL1PlOjOZR7+olVLOajyWNvhc7tgjovdgAZGjbFkjCG5E13v6K8vNTD6t2qgadKFtSzgwl8REhdMmKxw/jLzC1Bq4bVJ3CYTak4g47qpfLZvmTdi30K3aYQFEzLokgcJXmxZZi0tQg90EE+RNMfMyFgRxpacHHOgmxpvMZE= smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGLYopS5UbZEcsh1MaAdR9N9s0cRRdPmnLGpOGgbaFTBk49irNW34NokdIwBn4orfrTkahRgBoKRuY8Bbi6rc2H+SCKEN0QfIZyY3PypkI+UicQTSCSMHigiXXgFvuNKEpifNMIdTvOTPkxFVaBQdL76tB5Hr59v187h98ZOdhbgdaqRHF9WWGQ1sHA3G9fHwiVDmPmKK2OljIkk1n/nI/L6FCr03FykCXh5ImotKfng5r7obxssIezrWNm6GZgVVpoD4YWrp+FQR5LLNCb8kJv+16uolb9Jm6LD1eA4ufu49A4xmoCTY936oqvkpOJbiH7xRHyQ23KXl9ba8P+FQVrd+q4rttf1U3Zzmztv9I72R/EsVpScPlG1zLZWsha7E5hHiRDiru5DoHVOK/Wb/t4xMzw0TNXaA1V74JOvdovxbhZ87nfqmMjeBTFFBKwiGZEJFqBdTiSYtUuIdhpIJv4bd2QEDkOx0LbCMPDuqzMLdx80eOr5PoPXlhwtL1Mh8= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 3 chance_pgpnum_fix: 1 timeout: 1200 - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 0 - sudo ceph osd pool set cache min_write_recency_for_promote 0 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pool_snaps: true pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:22:09 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T06:18:36.255 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_61996d73ab738798e4ee0055c86457f5c459da4a/qa; will attempt to use it 2023-01-25T06:18:36.256 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_61996d73ab738798e4ee0055c86457f5c459da4a/qa/tasks 2023-01-25T06:18:36.257 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T06:18:36.258 INFO:teuthology.task.internal:Checking packages... 2023-01-25T06:18:36.285 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '61996d73ab738798e4ee0055c86457f5c459da4a' 2023-01-25T06:18:36.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T06:18:36.286 INFO:teuthology.packaging:ref: None 2023-01-25T06:18:36.286 INFO:teuthology.packaging:tag: None 2023-01-25T06:18:36.286 INFO:teuthology.packaging:branch: wip-yuri2-testing-2023-01-23-0928 2023-01-25T06:18:36.287 INFO:teuthology.packaging:sha1: 61996d73ab738798e4ee0055c86457f5c459da4a 2023-01-25T06:18:36.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri2-testing-2023-01-23-0928 2023-01-25T06:18:36.434 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1963.g61996d73 2023-01-25T06:18:36.435 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T06:18:36.442 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T06:18:36.442 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T06:18:36.463 INFO:teuthology.task.internal:Saving configuration 2023-01-25T06:18:36.482 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T06:18:36.489 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T06:18:36.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 06:13:16.339838', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpamKIdbtzcKH/eCGjeQCMBgseioAqdXQe/yGlYDbfjRXW5T9LXre0uYaNSituiaq12UivcGDjkHNUffsgyc9E='} 2023-01-25T06:18:36.538 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 06:13:16.338066', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBhaE+4y2wMi+sCiWhgCCWVNjp18eL2yUWeyzCXZtVruxqKXIqtjsC/W2BEp1B6mFH4NkZpghrH2z4dhcb3Slg4='} 2023-01-25T06:18:36.538 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T06:18:36.545 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-25T06:18:36.545 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-25T06:18:36.545 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T06:18:36.641 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb02f43b5e0>, signals=[15]) 2023-01-25T06:18:36.642 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T06:18:36.649 INFO:teuthology.task.internal:Opening connections... 2023-01-25T06:18:36.649 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:18:36.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:18:36.722 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:18:36.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:18:36.790 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T06:18:36.798 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-01-25T06:18:36.819 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-01-25T06:18:36.820 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-01-25T06:18:36.880 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-01-25T06:18:36.880 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8" 2023-01-25T06:18:36.880 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-01-25T06:18:36.881 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-01-25T06:18:36.881 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8" 2023-01-25T06:18:36.881 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-01-25T06:18:36.881 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T06:18:36.881 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-01-25T06:18:36.882 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T06:18:36.882 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-01-25T06:18:36.882 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T06:18:36.882 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T06:18:36.882 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T06:18:36.883 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-01-25T06:18:36.904 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2023-01-25T06:18:36.923 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2023-01-25T06:18:36.924 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2023-01-25T06:18:36.982 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2023-01-25T06:18:36.983 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8" 2023-01-25T06:18:36.983 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2023-01-25T06:18:36.983 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2023-01-25T06:18:36.983 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8" 2023-01-25T06:18:36.984 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2023-01-25T06:18:36.984 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T06:18:36.984 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2023-01-25T06:18:36.984 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T06:18:36.984 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2023-01-25T06:18:36.985 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T06:18:36.985 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T06:18:36.985 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T06:18:36.985 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2023-01-25T06:18:37.000 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T06:18:37.009 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T06:18:37.016 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T06:18:37.016 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2023-01-25T06:18:37.019 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2023-01-25T06:18:37.041 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T06:18:37.047 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T06:18:37.047 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2023-01-25T06:18:37.078 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2023-01-25T06:18:37.105 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T06:18:37.154 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T06:18:37.169 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T06:18:37.170 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T06:18:37.170 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T06:18:37.171 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T06:18:37.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:18:37.189 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2023-01-25T06:18:37.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:18:37.191 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-01-25T06:18:37.247 INFO:teuthology.orchestra.run.smithi110.stdout:4.18.0-408.el8.x86_64 2023-01-25T06:18:37.247 INFO:teuthology.task.kernel:Running kernel on smithi110: 4.18.0-408.el8.x86_64 2023-01-25T06:18:37.247 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2023-01-25T06:18:37.251 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-408.el8.x86_64 2023-01-25T06:18:37.251 INFO:teuthology.task.kernel:Running kernel on smithi062: 4.18.0-408.el8.x86_64 2023-01-25T06:18:37.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2023-01-25T06:18:40.768 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - AppStream 34 MB/s | 27 MB 00:00 2023-01-25T06:18:41.006 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - AppStream 33 MB/s | 27 MB 00:00 2023-01-25T06:18:46.656 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - BaseOS 41 MB/s | 27 MB 00:00 2023-01-25T06:18:46.777 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - BaseOS 37 MB/s | 27 MB 00:00 2023-01-25T06:18:50.888 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - Extras 173 kB/s | 18 kB 00:00 2023-01-25T06:18:51.276 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2023-01-25T06:18:51.299 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-25T06:18:51.661 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - PowerTools 23 MB/s | 5.6 MB 00:00 2023-01-25T06:18:52.523 INFO:teuthology.orchestra.run.smithi110.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-25T06:18:52.952 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 32 kB/s | 3.5 kB 00:00 2023-01-25T06:18:53.790 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2023-01-25T06:18:54.236 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2023-01-25T06:18:57.090 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 453 kB/s | 24 kB 00:00 2023-01-25T06:18:57.736 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 472 kB/s | 24 kB 00:00 2023-01-25T06:19:00.147 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T06:19:00.147 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T06:19:00.313 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:19:00.314 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:19:00.314 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:19:00.314 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:19:00.315 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:19:00.315 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T06:19:00.315 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T06:19:00.315 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T06:19:00.315 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:00.316 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:19:00.316 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:19:00.316 INFO:teuthology.orchestra.run.smithi110.stdout:Install 3 Packages 2023-01-25T06:19:00.316 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:00.317 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 85 M 2023-01-25T06:19:00.317 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 94 M 2023-01-25T06:19:00.317 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:19:00.617 INFO:teuthology.orchestra.run.smithi110.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 31 MB/s | 9.2 MB 00:00 2023-01-25T06:19:01.077 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T06:19:01.077 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T06:19:01.240 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:19:01.241 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:19:01.241 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:19:01.242 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:19:01.242 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:19:01.242 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T06:19:01.242 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T06:19:01.242 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T06:19:01.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:01.243 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:19:01.243 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:19:01.243 INFO:teuthology.orchestra.run.smithi062.stdout:Install 3 Packages 2023-01-25T06:19:01.243 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:01.244 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 85 M 2023-01-25T06:19:01.244 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 94 M 2023-01-25T06:19:01.245 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:19:01.480 INFO:teuthology.orchestra.run.smithi062.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 39 MB/s | 9.2 MB 00:00 2023-01-25T06:19:01.633 INFO:teuthology.orchestra.run.smithi110.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 31 MB/s | 41 MB 00:01 2023-01-25T06:19:02.084 INFO:teuthology.orchestra.run.smithi110.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 19 MB/s | 34 MB 00:01 2023-01-25T06:19:02.087 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:19:02.087 INFO:teuthology.orchestra.run.smithi110.stdout:Total 48 MB/s | 85 MB 00:01 2023-01-25T06:19:02.388 INFO:teuthology.orchestra.run.smithi062.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 36 MB/s | 41 MB 00:01 2023-01-25T06:19:02.723 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:19:02.963 INFO:teuthology.orchestra.run.smithi062.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 20 MB/s | 34 MB 00:01 2023-01-25T06:19:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:19:02.966 INFO:teuthology.orchestra.run.smithi062.stdout:Total 49 MB/s | 85 MB 00:01 2023-01-25T06:19:03.096 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:19:03.097 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:19:03.624 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:19:04.091 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:19:04.091 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:19:05.167 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:19:05.203 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:19:06.101 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:19:06.132 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:19:07.930 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:19:08.552 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:08.982 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:19:09.550 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:12.462 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:12.893 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:13.509 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:13.899 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:16.209 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:16.502 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:17.452 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:17.703 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:39.296 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:39.297 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:39.297 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:39.854 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:39.855 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:39.855 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:19:39.855 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T06:19:39.855 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T06:19:39.855 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:39.856 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:19:39.962 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel --last | head -n 1 2023-01-25T06:19:40.166 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 06:19:16 AM UTC 2023-01-25T06:19:40.166 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T06:19:40.167 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T06:19:40.167 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T06:19:40.168 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T06:19:40.168 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-01-25T06:19:40.168 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T06:19:40.169 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel --last | head -n 1 2023-01-25T06:19:40.357 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 06:19:16 AM UTC 2023-01-25T06:19:40.359 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T06:19:40.359 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2023-01-25T06:19:40.393 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2023-01-25T06:19:40.394 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2023-01-25T06:19:40.394 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.02 2023-01-25T06:19:40.394 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 129.el8 2023-01-25T06:19:40.395 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2023-01-25T06:19:40.395 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T06:19:40.395 INFO:teuthology.orchestra.run.smithi110.stdout:Group : System Environment/Base 2023-01-25T06:19:40.396 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 9265586 2023-01-25T06:19:40.396 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2023-01-25T06:19:40.397 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T06:19:40.397 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T06:19:40.397 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T06:19:40.398 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T06:19:40.398 INFO:teuthology.orchestra.run.smithi110.stdout:Relocations : (not relocatable) 2023-01-25T06:19:40.398 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : CentOS BuildSystem 2023-01-25T06:19:40.398 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2023-01-25T06:19:40.399 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T06:19:40.399 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2023-01-25T06:19:40.399 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2023-01-25T06:19:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:40.399 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T06:19:40.400 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T06:19:40.400 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T06:19:40.400 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2023-01-25T06:19:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:19:40.401 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T06:19:40.402 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T06:19:40.402 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 4.18.0-448.el8.x86_64 2023-01-25T06:19:40.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T06:19:41.013 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:41.040 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:41.472 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T06:19:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T06:19:41.736 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2023-01-25T06:19:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T06:19:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:42.544 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:19:42.544 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T06:19:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T06:19:42.545 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:42.545 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:19:42.648 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel --last | head -n 1 2023-01-25T06:19:42.853 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 06:19:17 AM UTC 2023-01-25T06:19:42.855 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T06:19:42.855 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T06:19:42.855 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T06:19:42.855 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T06:19:42.856 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-25T06:19:42.856 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi062.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T06:19:42.856 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel --last | head -n 1 2023-01-25T06:19:43.051 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 06:19:17 AM UTC 2023-01-25T06:19:43.053 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T06:19:43.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo rpm -qi grub2-tools 2023-01-25T06:19:43.088 INFO:teuthology.orchestra.run.smithi062.stdout:Name : grub2-tools 2023-01-25T06:19:43.088 INFO:teuthology.orchestra.run.smithi062.stdout:Epoch : 1 2023-01-25T06:19:43.088 INFO:teuthology.orchestra.run.smithi062.stdout:Version : 2.02 2023-01-25T06:19:43.088 INFO:teuthology.orchestra.run.smithi062.stdout:Release : 129.el8 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:Architecture: x86_64 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:Group : System Environment/Base 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:Size : 9265586 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:License : GPLv3+ 2023-01-25T06:19:43.089 INFO:teuthology.orchestra.run.smithi062.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T06:19:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T06:19:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T06:19:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T06:19:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:Relocations : (not relocatable) 2023-01-25T06:19:43.090 INFO:teuthology.orchestra.run.smithi062.stdout:Packager : CentOS BuildSystem 2023-01-25T06:19:43.091 INFO:teuthology.orchestra.run.smithi062.stdout:Vendor : CentOS 2023-01-25T06:19:43.091 INFO:teuthology.orchestra.run.smithi062.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T06:19:43.091 INFO:teuthology.orchestra.run.smithi062.stdout:Summary : Support tools for GRUB. 2023-01-25T06:19:43.091 INFO:teuthology.orchestra.run.smithi062.stdout:Description : 2023-01-25T06:19:43.091 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:43.092 INFO:teuthology.orchestra.run.smithi062.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T06:19:43.092 INFO:teuthology.orchestra.run.smithi062.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T06:19:43.092 INFO:teuthology.orchestra.run.smithi062.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T06:19:43.092 INFO:teuthology.orchestra.run.smithi062.stdout:hardware devices. 2023-01-25T06:19:43.093 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:19:43.093 INFO:teuthology.orchestra.run.smithi062.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T06:19:43.094 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T06:19:43.094 INFO:teuthology.task.kernel:Updating grub on smithi062 to boot 4.18.0-448.el8.x86_64 2023-01-25T06:19:43.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T06:19:43.939 INFO:teuthology.orchestra.run.smithi110.stderr:done 2023-01-25T06:19:43.941 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2023-01-25T06:19:43.973 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.Z2yp5Z7Xym 2023-01-25T06:19:43.973 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Z2yp5Z7Xym 2023-01-25T06:19:44.071 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.Z2yp5Z7Xym 2023-01-25T06:19:44.176 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.Z2yp5Z7Xym is 6KB 2023-01-25T06:19:44.225 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.Z2yp5Z7Xym 2023-01-25T06:19:44.242 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T06:19:44.316 INFO:teuthology.orchestra.run.smithi110.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T06:19:44.317 INFO:teuthology.orchestra.run.smithi110.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T06:19:44.317 INFO:teuthology.orchestra.run.smithi110.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T06:19:44.317 INFO:teuthology.orchestra.run.smithi110.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T06:19:44.319 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T06:19:44.386 INFO:teuthology.orchestra.run.smithi062.stderr:Generating grub configuration file ... 2023-01-25T06:19:44.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2023-01-25T06:19:46.168 INFO:teuthology.orchestra.run.smithi062.stderr:done 2023-01-25T06:19:46.169 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2023-01-25T06:19:46.212 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.5p1h1XD8cm 2023-01-25T06:19:46.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5p1h1XD8cm 2023-01-25T06:19:46.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0666 /tmp/tmp.5p1h1XD8cm 2023-01-25T06:19:46.445 DEBUG:teuthology.orchestra.remote:smithi062:/tmp/tmp.5p1h1XD8cm is 6KB 2023-01-25T06:19:46.493 DEBUG:teuthology.orchestra.run.smithi062:> rm -fr /tmp/tmp.5p1h1XD8cm 2023-01-25T06:19:46.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T06:19:46.629 INFO:teuthology.orchestra.run.smithi062.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T06:19:46.630 INFO:teuthology.orchestra.run.smithi062.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T06:19:46.630 INFO:teuthology.orchestra.run.smithi062.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T06:19:46.630 INFO:teuthology.orchestra.run.smithi062.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T06:19:46.631 DEBUG:teuthology.orchestra.run.smithi062:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T06:19:46.755 DEBUG:teuthology.orchestra.run.smithi062:> sudo shutdown -r now 2023-01-25T06:20:14.509 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-01-25T06:20:14.509 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-01-25T06:20:14.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:20:16.761 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-25T06:20:16.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-01-25T06:20:16.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:20:33.000 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2023-01-25T06:20:35.304 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-01-25T06:21:03.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-01-25T06:21:03.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:21:05.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-01-25T06:21:05.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:21:19.342 DEBUG:teuthology.orchestra.run.smithi110:> true 2023-01-25T06:21:19.755 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-01-25T06:21:19.755 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-01-25T06:21:19.756 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2023-01-25T06:21:19.773 INFO:teuthology.orchestra.run.smithi110.stdout:4.18.0-448.el8.x86_64 2023-01-25T06:21:19.774 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T06:21:19.774 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T06:21:19.775 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T06:21:20.776 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-25T06:21:20.777 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T06:21:20.941 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-01-25T06:21:21.164 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2023-01-25T06:21:21.190 DEBUG:teuthology.parallel:result is None 2023-01-25T06:21:21.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-01-25T06:21:21.305 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-25T06:21:21.305 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2023-01-25T06:21:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:4.18.0-448.el8.x86_64 2023-01-25T06:21:21.324 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T06:21:21.324 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T06:21:21.324 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T06:21:22.325 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T06:21:22.325 DEBUG:teuthology.orchestra.run.smithi062:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T06:21:22.421 INFO:teuthology.orchestra.run.smithi062.stdout:ttyS1 2023-01-25T06:21:22.445 DEBUG:teuthology.parallel:result is None 2023-01-25T06:21:22.446 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T06:21:22.452 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T06:21:22.453 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T06:21:22.455 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T06:21:22.476 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T06:21:22.485 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T06:21:22.492 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T06:21:22.492 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T06:21:22.515 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T06:21:22.558 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T06:21:22.568 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T06:21:22.569 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T06:21:22.581 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T06:21:22.617 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:21:22.629 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:21:22.648 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:21:22.659 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T06:21:22.660 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T06:21:22.670 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T06:21:22.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T06:21:22.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T06:21:22.708 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T06:21:22.719 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T06:21:22.720 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T06:21:22.747 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T06:21:22.766 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T06:21:22.847 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T06:21:22.892 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T06:21:22.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T06:21:23.020 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:21:23.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T06:21:23.089 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T06:21:23.134 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T06:21:23.181 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T06:21:23.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T06:21:23.309 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:21:23.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T06:21:23.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2023-01-25T06:21:23.384 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2023-01-25T06:21:23.444 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T06:21:23.488 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T06:21:24.421 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T06:21:24.430 INFO:teuthology.task.internal:Starting timer... 2023-01-25T06:21:24.430 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T06:21:24.443 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T06:21:24.452 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T06:21:24.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2023-01-25T06:21:24.507 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2023-01-25T06:21:24.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2023-01-25T06:21:24.606 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2023-01-25T06:21:24.608 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T06:21:24.608 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2023-01-25T06:21:24.627 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2023-01-25T06:21:24.628 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2023-01-25T06:21:24.661 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2023-01-25T06:21:24.661 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive'} 2023-01-25T06:21:24.662 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T06:21:24.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:21:24.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T06:21:24.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T06:21:24.729 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T06:21:24.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2023-01-25T06:21:24.776 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2023-01-25T06:21:24.823 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T06:21:24.835 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T06:21:24.836 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T06:21:24.857 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T06:21:24.859 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T06:27:59.775 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com')] 2023-01-25T06:27:59.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-01-25T06:27:59.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:27:59.848 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-01-25T06:27:59.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-01-25T06:27:59.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T06:27:59.986 DEBUG:teuthology.orchestra.run.smithi110:> true 2023-01-25T06:28:00.058 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T06:28:00.069 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T06:28:00.069 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T06:28:00.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T06:28:00.072 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T06:28:00.073 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T06:28:00.104 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T06:28:00.118 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T06:28:00.130 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T06:28:00.144 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T06:28:00.186 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2023-01-25T06:28:00.197 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2023-01-25T06:28:00.212 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T06:28:00.213 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2023-01-25T06:28:00.224 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2023-01-25T06:28:00.226 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T06:28:00.240 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T06:28:00.252 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T06:28:00.296 INFO:teuthology.orchestra.run.smithi062.stderr:bash: ntpq: command not found 2023-01-25T06:28:00.300 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T06:28:00.300 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2023-01-25T06:28:00.301 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.301 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.301 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.301 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.318 INFO:teuthology.orchestra.run.smithi110.stderr:bash: ntpq: command not found 2023-01-25T06:28:00.322 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T06:28:00.322 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2023-01-25T06:28:00.322 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.322 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.323 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.323 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T06:28:00.324 INFO:teuthology.run_tasks:Running task install... 2023-01-25T06:28:00.333 DEBUG:teuthology.task.install:project ceph 2023-01-25T06:28:00.333 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}} 2023-01-25T06:28:00.333 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'} 2023-01-25T06:28:00.334 INFO:teuthology.task.install:Using flavor: default 2023-01-25T06:28:00.339 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T06:28:00.340 INFO:teuthology.task.install:extra packages: [] 2023-01-25T06:28:00.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-25T06:28:00.380 INFO:teuthology.orchestra.run.smithi062.stdout:CentOSStream 2023-01-25T06:28:00.381 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:28:00.382 DEBUG:teuthology.orchestra.run.smithi110:> sudo lsb_release -is 2023-01-25T06:28:00.382 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a', 'tag': None, 'wait_for_package': False} 2023-01-25T06:28:00.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-25T06:28:00.420 INFO:teuthology.orchestra.run.smithi110.stdout:CentOSStream 2023-01-25T06:28:00.422 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:28:00.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a', 'tag': None, 'wait_for_package': False} 2023-01-25T06:28:00.423 DEBUG:teuthology.orchestra.run.smithi110:> sudo lsb_release -is 2023-01-25T06:28:00.460 INFO:teuthology.orchestra.run.smithi062.stdout:CentOSStream 2023-01-25T06:28:00.461 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:28:00.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=61996d73ab738798e4ee0055c86457f5c459da4a 2023-01-25T06:28:00.506 INFO:teuthology.orchestra.run.smithi110.stdout:CentOSStream 2023-01-25T06:28:00.507 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:28:00.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=61996d73ab738798e4ee0055c86457f5c459da4a 2023-01-25T06:28:00.660 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/ 2023-01-25T06:28:00.661 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1963.g61996d73 2023-01-25T06:28:00.710 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/ 2023-01-25T06:28:00.710 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1963.g61996d73 2023-01-25T06:28:00.862 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T06:28:00.863 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:28:00.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T06:28:00.891 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T06:28:00.892 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/61996d73ab738798e4ee0055c86457f5c459da4a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T06:28:00.919 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri2-testing-2023-01-23-0928/61996d73ab738798e4ee0055c86457f5c459da4a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T06:28:00.919 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:28:00.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T06:28:00.947 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T06:28:00.947 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/61996d73ab738798e4ee0055c86457f5c459da4a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T06:28:00.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T06:28:01.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T06:28:01.039 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T06:28:01.092 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T06:28:01.107 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2023-01-25T06:28:01.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2023-01-25T06:28:01.159 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2023-01-25T06:28:01.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2023-01-25T06:28:01.367 INFO:teuthology.orchestra.run.smithi062.stdout:61 files removed 2023-01-25T06:28:01.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw 2023-01-25T06:28:01.420 INFO:teuthology.orchestra.run.smithi110.stdout:61 files removed 2023-01-25T06:28:01.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw 2023-01-25T06:28:02.420 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 134 kB/s | 80 kB 00:00 2023-01-25T06:28:02.458 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 132 kB/s | 80 kB 00:00 2023-01-25T06:28:02.987 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-01-25T06:28:03.242 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-01-25T06:28:03.455 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 3.4 kB/s | 1.5 kB 00:00 2023-01-25T06:28:03.902 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 2.4 kB/s | 1.5 kB 00:00 2023-01-25T06:28:04.249 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - AppStream 36 MB/s | 27 MB 00:00 2023-01-25T06:28:04.745 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - AppStream 34 MB/s | 27 MB 00:00 2023-01-25T06:28:10.505 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - BaseOS 36 MB/s | 27 MB 00:00 2023-01-25T06:28:10.725 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - BaseOS 31 MB/s | 27 MB 00:00 2023-01-25T06:28:14.771 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2023-01-25T06:28:15.156 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS-8 - PowerTools 22 MB/s | 5.6 MB 00:00 2023-01-25T06:28:15.482 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - Extras 152 kB/s | 18 kB 00:00 2023-01-25T06:28:15.892 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T06:28:16.397 INFO:teuthology.orchestra.run.smithi110.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T06:28:17.216 INFO:teuthology.orchestra.run.smithi062.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-01-25T06:28:18.709 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 13 MB 00:01 2023-01-25T06:28:18.822 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 13 MB 00:02 2023-01-25T06:28:22.148 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 495 kB/s | 24 kB 00:00 2023-01-25T06:28:22.268 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 392 kB/s | 24 kB 00:00 2023-01-25T06:28:25.564 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:28:25.567 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:28:25.567 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:28:25.567 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:28:25.568 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:28:25.568 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:18.0.0-1963.g61996d73.el8 ceph 13 M 2023-01-25T06:28:25.568 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2023-01-25T06:28:25.568 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.6 M 2023-01-25T06:28:25.569 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.6 M 2023-01-25T06:28:25.569 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:28:25.569 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:18.0.0-1963.g61996d73.el8 ceph 5.2 M 2023-01-25T06:28:25.569 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:18.0.0-1963.g61996d73.el8 ceph 24 M 2023-01-25T06:28:25.570 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:18.0.0-1963.g61996d73.el8 ceph 24 k 2023-01-25T06:28:25.570 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T06:28:25.570 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 767 k 2023-01-25T06:28:25.570 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T06:28:25.571 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T06:28:25.571 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 474 k 2023-01-25T06:28:25.571 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T06:28:25.571 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 6.5 M 2023-01-25T06:28:25.571 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T06:28:25.572 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T06:28:25.572 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse 2023-01-25T06:28:25.572 INFO:teuthology.orchestra.run.smithi110.stdout: x86_64 2:18.0.0-1963.g61996d73.el8 ceph 45 k 2023-01-25T06:28:25.572 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:18.0.0-1963.g61996d73.el8 ceph 115 k 2023-01-25T06:28:25.572 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:18.0.0-1963.g61996d73.el8 ceph 190 k 2023-01-25T06:28:25.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T06:28:25.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:18.0.0-1963.g61996d73.el8 ceph 371 k 2023-01-25T06:28:25.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 365 k 2023-01-25T06:28:25.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:18.0.0-1963.g61996d73.el8 ceph 107 k 2023-01-25T06:28:25.574 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T06:28:25.574 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:28:25.574 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:28:25.574 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:28:25.575 INFO:teuthology.orchestra.run.smithi110.stdout:Install 21 Packages 2023-01-25T06:28:25.575 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 2 Packages 2023-01-25T06:28:25.575 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:28:25.576 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 62 M 2023-01-25T06:28:25.576 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:28:25.953 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:28:25.956 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:28:25.956 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:28:25.957 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:28:25.957 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:28:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:18.0.0-1963.g61996d73.el8 ceph 13 M 2023-01-25T06:28:25.957 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2023-01-25T06:28:25.957 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.6 M 2023-01-25T06:28:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.6 M 2023-01-25T06:28:25.958 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:28:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:18.0.0-1963.g61996d73.el8 ceph 5.2 M 2023-01-25T06:28:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:18.0.0-1963.g61996d73.el8 ceph 24 M 2023-01-25T06:28:25.958 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:18.0.0-1963.g61996d73.el8 ceph 24 k 2023-01-25T06:28:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T06:28:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 767 k 2023-01-25T06:28:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T06:28:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T06:28:25.959 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 474 k 2023-01-25T06:28:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T06:28:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:18.0.0-1963.g61996d73.el8 ceph 6.5 M 2023-01-25T06:28:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T06:28:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T06:28:25.960 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse 2023-01-25T06:28:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: x86_64 2:18.0.0-1963.g61996d73.el8 ceph 45 k 2023-01-25T06:28:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:18.0.0-1963.g61996d73.el8 ceph 115 k 2023-01-25T06:28:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:18.0.0-1963.g61996d73.el8 ceph 190 k 2023-01-25T06:28:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T06:28:25.961 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:18.0.0-1963.g61996d73.el8 ceph 371 k 2023-01-25T06:28:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 365 k 2023-01-25T06:28:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:18.0.0-1963.g61996d73.el8 ceph 107 k 2023-01-25T06:28:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T06:28:25.962 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:28:25.962 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:28:25.963 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:28:25.963 INFO:teuthology.orchestra.run.smithi062.stdout:Install 21 Packages 2023-01-25T06:28:25.963 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 2 Packages 2023-01-25T06:28:25.963 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:28:25.964 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 62 M 2023-01-25T06:28:25.964 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:28:26.341 INFO:teuthology.orchestra.run.smithi110.stdout:(1/23): ceph-base-18.0.0-1963.g61996d73.el8.x86 6.8 MB/s | 5.2 MB 00:00 2023-01-25T06:28:26.433 INFO:teuthology.orchestra.run.smithi110.stdout:(2/23): ceph-selinux-18.0.0-1963.g61996d73.el8. 266 kB/s | 24 kB 00:00 2023-01-25T06:28:26.566 INFO:teuthology.orchestra.run.smithi110.stdout:(3/23): libcephfs2-18.0.0-1963.g61996d73.el8.x8 5.6 MB/s | 767 kB 00:00 2023-01-25T06:28:26.667 INFO:teuthology.orchestra.run.smithi110.stdout:(4/23): libradosstriper1-18.0.0-1963.g61996d73. 4.6 MB/s | 474 kB 00:00 2023-01-25T06:28:27.750 INFO:teuthology.orchestra.run.smithi110.stdout:(5/23): librgw2-18.0.0-1963.g61996d73.el8.x86_6 6.0 MB/s | 6.5 MB 00:01 2023-01-25T06:28:27.809 INFO:teuthology.orchestra.run.smithi110.stdout:(6/23): python3-ceph-argparse-18.0.0-1963.g6199 765 kB/s | 45 kB 00:00 2023-01-25T06:28:27.951 INFO:teuthology.orchestra.run.smithi110.stdout:(7/23): ceph-radosgw-18.0.0-1963.g61996d73.el8. 5.5 MB/s | 13 MB 00:02 2023-01-25T06:28:27.976 INFO:teuthology.orchestra.run.smithi110.stdout:(8/23): python3-ceph-common-18.0.0-1963.g61996d 687 kB/s | 115 kB 00:00 2023-01-25T06:28:28.010 INFO:teuthology.orchestra.run.smithi110.stdout:(9/23): python3-cephfs-18.0.0-1963.g61996d73.el 3.2 MB/s | 190 kB 00:00 2023-01-25T06:28:28.077 INFO:teuthology.orchestra.run.smithi110.stdout:(10/23): python3-rados-18.0.0-1963.g61996d73.el 3.6 MB/s | 371 kB 00:00 2023-01-25T06:28:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:(11/23): python3-rbd-18.0.0-1963.g61996d73.el8. 3.9 MB/s | 365 kB 00:00 2023-01-25T06:28:28.127 INFO:teuthology.orchestra.run.smithi110.stdout:(12/23): python3-rgw-18.0.0-1963.g61996d73.el8. 2.1 MB/s | 107 kB 00:00 2023-01-25T06:28:28.203 INFO:teuthology.orchestra.run.smithi110.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-01-25T06:28:28.228 INFO:teuthology.orchestra.run.smithi110.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-01-25T06:28:28.253 INFO:teuthology.orchestra.run.smithi110.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-25T06:28:28.484 INFO:teuthology.orchestra.run.smithi110.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 736 kB/s | 259 kB 00:00 2023-01-25T06:28:28.521 INFO:teuthology.orchestra.run.smithi110.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-01-25T06:28:28.579 INFO:teuthology.orchestra.run.smithi110.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-01-25T06:28:28.655 INFO:teuthology.orchestra.run.smithi110.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 346 kB/s | 59 kB 00:00 2023-01-25T06:28:28.913 INFO:teuthology.orchestra.run.smithi110.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 5.2 MB/s | 1.7 MB 00:00 2023-01-25T06:28:29.188 INFO:teuthology.orchestra.run.smithi110.stdout:(21/23): librados2-18.0.0-1963.g61996d73.el8.x8 6.8 MB/s | 3.6 MB 00:00 2023-01-25T06:28:29.385 INFO:teuthology.orchestra.run.smithi062.stdout:(1/23): ceph-radosgw-18.0.0-1963.g61996d73.el8. 3.8 MB/s | 13 MB 00:03 2023-01-25T06:28:29.436 INFO:teuthology.orchestra.run.smithi062.stdout:(2/23): ceph-selinux-18.0.0-1963.g61996d73.el8. 489 kB/s | 24 kB 00:00 2023-01-25T06:28:29.573 INFO:teuthology.orchestra.run.smithi110.stdout:(22/23): librbd1-18.0.0-1963.g61996d73.el8.x86_ 5.4 MB/s | 3.6 MB 00:00 2023-01-25T06:28:29.594 INFO:teuthology.orchestra.run.smithi062.stdout:(3/23): libcephfs2-18.0.0-1963.g61996d73.el8.x8 4.7 MB/s | 767 kB 00:00 2023-01-25T06:28:29.719 INFO:teuthology.orchestra.run.smithi062.stdout:(4/23): libradosstriper1-18.0.0-1963.g61996d73. 3.7 MB/s | 474 kB 00:00 2023-01-25T06:28:29.881 INFO:teuthology.orchestra.run.smithi110.stdout:(23/23): ceph-common-18.0.0-1963.g61996d73.el8. 5.7 MB/s | 24 MB 00:04 2023-01-25T06:28:29.885 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:28:29.885 INFO:teuthology.orchestra.run.smithi110.stdout:Total 14 MB/s | 62 MB 00:04 2023-01-25T06:28:29.924 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:28:29.969 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:28:29.969 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:28:30.470 INFO:teuthology.orchestra.run.smithi062.stdout:(5/23): ceph-base-18.0.0-1963.g61996d73.el8.x86 1.2 MB/s | 5.2 MB 00:04 2023-01-25T06:28:30.537 INFO:teuthology.orchestra.run.smithi062.stdout:(6/23): python3-ceph-argparse-18.0.0-1963.g6199 671 kB/s | 45 kB 00:00 2023-01-25T06:28:30.591 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:28:30.594 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:28:30.645 INFO:teuthology.orchestra.run.smithi062.stdout:(7/23): python3-ceph-common-18.0.0-1963.g61996d 1.0 MB/s | 115 kB 00:00 2023-01-25T06:28:30.770 INFO:teuthology.orchestra.run.smithi062.stdout:(8/23): python3-cephfs-18.0.0-1963.g61996d73.el 1.5 MB/s | 190 kB 00:00 2023-01-25T06:28:30.954 INFO:teuthology.orchestra.run.smithi062.stdout:(9/23): librgw2-18.0.0-1963.g61996d73.el8.x86_6 5.3 MB/s | 6.5 MB 00:01 2023-01-25T06:28:31.021 INFO:teuthology.orchestra.run.smithi062.stdout:(10/23): python3-rados-18.0.0-1963.g61996d73.el 1.4 MB/s | 371 kB 00:00 2023-01-25T06:28:31.088 INFO:teuthology.orchestra.run.smithi062.stdout:(11/23): python3-rbd-18.0.0-1963.g61996d73.el8. 2.7 MB/s | 365 kB 00:00 2023-01-25T06:28:31.147 INFO:teuthology.orchestra.run.smithi062.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-01-25T06:28:31.172 INFO:teuthology.orchestra.run.smithi062.stdout:(13/23): python3-rgw-18.0.0-1963.g61996d73.el8. 708 kB/s | 107 kB 00:00 2023-01-25T06:28:31.206 INFO:teuthology.orchestra.run.smithi062.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-01-25T06:28:31.231 INFO:teuthology.orchestra.run.smithi062.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.0 MB/s | 47 kB 00:00 2023-01-25T06:28:31.257 INFO:teuthology.orchestra.run.smithi062.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 522 kB/s | 44 kB 00:00 2023-01-25T06:28:31.715 INFO:teuthology.orchestra.run.smithi062.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 128 kB/s | 59 kB 00:00 2023-01-25T06:28:31.805 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:28:31.932 INFO:teuthology.orchestra.run.smithi062.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 346 kB/s | 75 kB 00:00 2023-01-25T06:28:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T06:28:32.242 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T06:28:32.384 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T06:28:32.762 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T06:28:32.790 INFO:teuthology.orchestra.run.smithi062.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 196 kB/s | 306 kB 00:01 2023-01-25T06:28:32.959 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:28:33.024 INFO:teuthology.orchestra.run.smithi062.stdout:(20/23): ceph-common-18.0.0-1963.g61996d73.el8. 3.5 MB/s | 24 MB 00:07 2023-01-25T06:28:33.059 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:28:33.323 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 5/25 2023-01-25T06:28:33.500 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 6/25 2023-01-25T06:28:33.899 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 6/25 2023-01-25T06:28:34.059 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:28:34.074 INFO:teuthology.orchestra.run.smithi062.stdout:(21/23): librbd1-18.0.0-1963.g61996d73.el8.x86_ 3.4 MB/s | 3.6 MB 00:01 2023-01-25T06:28:34.129 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:28:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T06:28:34.442 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T06:28:34.498 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T06:28:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8. 10/25 2023-01-25T06:28:34.860 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 11/25 2023-01-25T06:28:35.083 INFO:teuthology.orchestra.run.smithi062.stdout:(22/23): thrift-0.13.0-2.el8.x86_64.rpm 564 kB/s | 1.7 MB 00:03 2023-01-25T06:28:35.241 INFO:teuthology.orchestra.run.smithi062.stdout:(23/23): librados2-18.0.0-1963.g61996d73.el8.x8 1.5 MB/s | 3.6 MB 00:02 2023-01-25T06:28:35.245 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:28:35.246 INFO:teuthology.orchestra.run.smithi062.stdout:Total 6.6 MB/s | 62 MB 00:09 2023-01-25T06:28:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 12/25 2023-01-25T06:28:35.289 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:28:35.338 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:28:35.338 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:28:35.433 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 13/25 2023-01-25T06:28:35.504 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 13/25 2023-01-25T06:28:35.692 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T06:28:35.980 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:28:35.984 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:28:36.519 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T06:28:36.725 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 16/25 2023-01-25T06:28:36.801 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 16/25 2023-01-25T06:28:37.024 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 17/25 2023-01-25T06:28:37.207 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T06:28:37.231 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:28:37.409 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x8 19/25 2023-01-25T06:28:37.418 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T06:28:37.617 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T06:28:37.778 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T06:28:38.185 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T06:28:38.370 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:28:38.475 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:28:38.730 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 5/25 2023-01-25T06:28:38.895 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 6/25 2023-01-25T06:28:39.313 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 6/25 2023-01-25T06:28:39.462 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:28:39.531 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:28:39.712 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T06:28:39.837 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T06:28:39.909 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T06:28:40.044 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8. 10/25 2023-01-25T06:28:40.249 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 11/25 2023-01-25T06:28:40.467 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 12/25 2023-01-25T06:28:40.595 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 13/25 2023-01-25T06:28:40.656 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 13/25 2023-01-25T06:28:40.839 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T06:28:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:41.679 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T06:28:41.869 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 16/25 2023-01-25T06:28:41.951 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 16/25 2023-01-25T06:28:41.982 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:42.153 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 17/25 2023-01-25T06:28:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 21/25 2023-01-25T06:28:42.420 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 21/25 2023-01-25T06:28:42.492 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T06:28:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:28:42.670 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x8 19/25 2023-01-25T06:28:46.620 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:46.844 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:47.494 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 20/25 2023-01-25T06:28:47.787 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 21/25 2023-01-25T06:28:47.977 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 21/25 2023-01-25T06:28:48.278 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:29:01.608 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:29:01.608 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2023-01-25T06:29:01.608 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2023-01-25T06:29:01.608 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /dev 2023-01-25T06:29:01.609 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /run 2023-01-25T06:29:01.609 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2023-01-25T06:29:01.609 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2023-01-25T06:29:01.609 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2023-01-25T06:29:01.609 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2023-01-25T06:29:01.610 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /dev 2023-01-25T06:29:01.610 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:03.026 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 23/25 2023-01-25T06:29:03.047 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 23/25 2023-01-25T06:29:03.047 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T06:29:03.266 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T06:29:03.266 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 1/25 2023-01-25T06:29:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2/25 2023-01-25T06:29:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 3/25 2023-01-25T06:29:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:29:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 5/25 2023-01-25T06:29:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 6/25 2023-01-25T06:29:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:29:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8. 8/25 2023-01-25T06:29:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x8 9/25 2023-01-25T06:29:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 10/25 2023-01-25T06:29:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 11/25 2023-01-25T06:29:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 12/25 2023-01-25T06:29:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 13/25 2023-01-25T06:29:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T06:29:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T06:29:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T06:29:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T06:29:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T06:29:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T06:29:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T06:29:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T06:29:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:29:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T06:29:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 24/25 2023-01-25T06:29:04.834 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:04.834 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:04.835 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2023-01-25T06:29:04.835 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.835 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.835 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:04.835 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.836 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.836 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.836 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.836 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T06:29:04.837 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.837 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T06:29:04.837 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T06:29:04.837 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.837 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T06:29:04.838 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.838 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T06:29:04.838 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T06:29:04.838 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.838 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.839 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.839 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T06:29:04.839 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.839 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.839 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:04.840 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T06:29:04.840 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:04.840 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:05.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-test 2023-01-25T06:29:05.866 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:43 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:07.014 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:07.015 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:07.015 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:29:07.015 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:07.015 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:07.015 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:18.0.0-1963.g61996d73.el8 ceph 47 M 2023-01-25T06:29:07.016 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:29:07.016 INFO:teuthology.orchestra.run.smithi110.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T06:29:07.016 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:18.0.0-1963.g61996d73.el8 ceph 137 k 2023-01-25T06:29:07.016 INFO:teuthology.orchestra.run.smithi110.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T06:29:07.017 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T06:29:07.017 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T06:29:07.017 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:07.017 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:07.017 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:07.018 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2023-01-25T06:29:07.018 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:07.018 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 48 M 2023-01-25T06:29:07.019 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 218 M 2023-01-25T06:29:07.019 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:29:07.081 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 3.1 MB/s | 202 kB 00:00 2023-01-25T06:29:07.115 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-01-25T06:29:07.148 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-01-25T06:29:07.174 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-01-25T06:29:07.491 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): libcephsqlite-18.0.0-1963.g61996d73.el8. 290 kB/s | 137 kB 00:00 2023-01-25T06:29:08.224 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:29:08.224 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2023-01-25T06:29:08.224 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2023-01-25T06:29:08.225 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-01-25T06:29:08.225 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /run 2023-01-25T06:29:08.225 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2023-01-25T06:29:08.225 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2023-01-25T06:29:08.225 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2023-01-25T06:29:08.226 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2023-01-25T06:29:08.226 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /dev 2023-01-25T06:29:08.226 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:09.683 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 23/25 2023-01-25T06:29:09.705 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 23/25 2023-01-25T06:29:09.705 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T06:29:09.910 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T06:29:09.911 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:10.608 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:10.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 1/25 2023-01-25T06:29:10.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2/25 2023-01-25T06:29:10.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 3/25 2023-01-25T06:29:10.609 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 4/25 2023-01-25T06:29:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 5/25 2023-01-25T06:29:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_6 6/25 2023-01-25T06:29:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 7/25 2023-01-25T06:29:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8. 8/25 2023-01-25T06:29:10.610 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x8 9/25 2023-01-25T06:29:10.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 10/25 2023-01-25T06:29:10.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 11/25 2023-01-25T06:29:10.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 12/25 2023-01-25T06:29:10.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 13/25 2023-01-25T06:29:10.611 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T06:29:10.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T06:29:10.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T06:29:10.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T06:29:10.612 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T06:29:10.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T06:29:10.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T06:29:10.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T06:29:10.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:18.0.0-1963.g61996d73.el8.x86_64 22/25 2023-01-25T06:29:10.613 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T06:29:10.614 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 24/25 2023-01-25T06:29:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T06:29:11.493 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:11.494 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2023-01-25T06:29:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.494 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:29:11.494 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T06:29:11.495 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T06:29:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T06:29:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T06:29:11.496 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T06:29:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T06:29:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.497 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T06:29:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.498 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T06:29:11.499 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:11.499 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:29:11.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-test 2023-01-25T06:29:11.957 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): ceph-test-18.0.0-1963.g61996d73.el8.x86_ 9.6 MB/s | 47 MB 00:04 2023-01-25T06:29:11.960 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:11.961 INFO:teuthology.orchestra.run.smithi110.stdout:Total 9.8 MB/s | 48 MB 00:04 2023-01-25T06:29:12.003 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:29:12.024 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:29:12.024 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:29:12.326 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:29:12.329 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:29:12.545 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:50 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:12.789 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:29:13.011 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T06:29:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T06:29:13.329 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T06:29:13.395 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T06:29:13.580 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T06:29:13.737 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 5/6 2023-01-25T06:29:13.779 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:29:13.780 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:13.780 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:29:13.781 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:13.781 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:29:13.781 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:18.0.0-1963.g61996d73.el8 ceph 47 M 2023-01-25T06:29:13.781 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:29:13.781 INFO:teuthology.orchestra.run.smithi062.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T06:29:13.782 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:18.0.0-1963.g61996d73.el8 ceph 137 k 2023-01-25T06:29:13.782 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T06:29:13.782 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T06:29:13.782 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T06:29:13.783 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:13.783 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:29:13.783 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:13.783 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2023-01-25T06:29:13.783 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:13.784 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 48 M 2023-01-25T06:29:13.784 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 218 M 2023-01-25T06:29:13.784 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:29:13.855 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.8 MB/s | 202 kB 00:00 2023-01-25T06:29:13.872 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-01-25T06:29:13.906 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-01-25T06:29:13.931 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-01-25T06:29:14.102 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): libcephsqlite-18.0.0-1963.g61996d73.el8. 431 kB/s | 137 kB 00:00 2023-01-25T06:29:18.315 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 5/6 2023-01-25T06:29:18.616 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 6/6 2023-01-25T06:29:18.918 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 6/6 2023-01-25T06:29:18.919 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 1/6 2023-01-25T06:29:18.919 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 2/6 2023-01-25T06:29:18.919 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T06:29:18.919 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T06:29:18.920 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T06:29:19.444 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T06:29:19.445 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:19.445 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:19.445 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:19.445 INFO:teuthology.orchestra.run.smithi110.stdout: jq-1.6-6.el8.x86_64 2023-01-25T06:29:19.446 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:19.446 INFO:teuthology.orchestra.run.smithi110.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T06:29:19.446 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T06:29:19.446 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T06:29:19.447 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:19.447 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:19.667 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph 2023-01-25T06:29:20.400 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:21.549 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:21.552 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2023-01-25T06:29:21.552 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:29:21.552 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2023-01-25T06:29:21.553 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:21.553 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:18.0.0-1963.g61996d73.el8 ceph 6.4 k 2023-01-25T06:29:21.553 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:29:21.553 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:18.0.0-1963.g61996d73.el8 ceph 2.3 M 2023-01-25T06:29:21.553 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:18.0.0-1963.g61996d73.el8 ceph 1.5 M 2023-01-25T06:29:21.554 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 236 k 2023-01-25T06:29:21.554 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.9 M 2023-01-25T06:29:21.554 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 17 M 2023-01-25T06:29:21.554 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T06:29:21.554 INFO:teuthology.orchestra.run.smithi110.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T06:29:21.555 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T06:29:21.555 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T06:29:21.555 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T06:29:21.556 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T06:29:21.556 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T06:29:21.556 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T06:29:21.556 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T06:29:21.556 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T06:29:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T06:29:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T06:29:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T06:29:21.557 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T06:29:21.558 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T06:29:21.558 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T06:29:21.558 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T06:29:21.558 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T06:29:21.558 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T06:29:21.559 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T06:29:21.559 INFO:teuthology.orchestra.run.smithi110.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T06:29:21.559 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T06:29:21.559 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T06:29:21.560 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T06:29:21.560 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T06:29:21.560 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T06:29:21.560 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:21.561 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:21.561 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2023-01-25T06:29:21.561 INFO:teuthology.orchestra.run.smithi110.stdout:Install 32 Packages 2023-01-25T06:29:21.561 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:21.562 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 28 M 2023-01-25T06:29:21.562 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 110 M 2023-01-25T06:29:21.562 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:29:21.819 INFO:teuthology.orchestra.run.smithi110.stdout:(1/32): ceph-18.0.0-1963.g61996d73.el8.x86_64.r 25 kB/s | 6.4 kB 00:00 2023-01-25T06:29:22.894 INFO:teuthology.orchestra.run.smithi110.stdout:(2/32): ceph-mgr-18.0.0-1963.g61996d73.el8.x86_ 1.1 MB/s | 1.5 MB 00:01 2023-01-25T06:29:23.220 INFO:teuthology.orchestra.run.smithi110.stdout:(3/32): ceph-mds-18.0.0-1963.g61996d73.el8.x86_ 1.4 MB/s | 2.3 MB 00:01 2023-01-25T06:29:23.361 INFO:teuthology.orchestra.run.smithi110.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1963.g6199 1.6 MB/s | 236 kB 00:00 2023-01-25T06:29:23.420 INFO:teuthology.orchestra.run.smithi110.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T06:29:23.437 INFO:teuthology.orchestra.run.smithi110.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 2.3 MB/s | 39 kB 00:00 2023-01-25T06:29:23.462 INFO:teuthology.orchestra.run.smithi110.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-01-25T06:29:23.479 INFO:teuthology.orchestra.run.smithi110.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 3.2 MB/s | 54 kB 00:00 2023-01-25T06:29:23.513 INFO:teuthology.orchestra.run.smithi110.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-01-25T06:29:23.538 INFO:teuthology.orchestra.run.smithi110.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-01-25T06:29:23.589 INFO:teuthology.orchestra.run.smithi110.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 11 MB/s | 559 kB 00:00 2023-01-25T06:29:23.614 INFO:teuthology.orchestra.run.smithi110.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T06:29:23.848 INFO:teuthology.orchestra.run.smithi110.stdout:(13/32): ceph-mon-18.0.0-1963.g61996d73.el8.x86 1.9 MB/s | 3.9 MB 00:02 2023-01-25T06:29:23.931 INFO:teuthology.orchestra.run.smithi110.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 137 kB/s | 44 kB 00:00 2023-01-25T06:29:24.132 INFO:teuthology.orchestra.run.smithi110.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 863 kB/s | 173 kB 00:00 2023-01-25T06:29:24.182 INFO:teuthology.orchestra.run.smithi110.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 554 kB/s | 185 kB 00:00 2023-01-25T06:29:24.266 INFO:teuthology.orchestra.run.smithi110.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-01-25T06:29:24.350 INFO:teuthology.orchestra.run.smithi110.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2023-01-25T06:29:24.466 INFO:teuthology.orchestra.run.smithi110.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 418 kB/s | 49 kB 00:00 2023-01-25T06:29:24.492 INFO:teuthology.orchestra.run.smithi110.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.0 MB/s | 384 kB 00:00 2023-01-25T06:29:24.551 INFO:teuthology.orchestra.run.smithi110.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 701 kB/s | 59 kB 00:00 2023-01-25T06:29:24.626 INFO:teuthology.orchestra.run.smithi110.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-01-25T06:29:24.659 INFO:teuthology.orchestra.run.smithi110.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-01-25T06:29:24.693 INFO:teuthology.orchestra.run.smithi110.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 280 kB/s | 19 kB 00:00 2023-01-25T06:29:24.727 INFO:teuthology.orchestra.run.smithi110.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 357 kB/s | 24 kB 00:00 2023-01-25T06:29:24.760 INFO:teuthology.orchestra.run.smithi110.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 436 kB/s | 29 kB 00:00 2023-01-25T06:29:24.819 INFO:teuthology.orchestra.run.smithi110.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 299 kB/s | 27 kB 00:00 2023-01-25T06:29:24.928 INFO:teuthology.orchestra.run.smithi110.stdout:(28/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.4 MB/s | 241 kB 00:00 2023-01-25T06:29:24.978 INFO:teuthology.orchestra.run.smithi110.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2023-01-25T06:29:25.003 INFO:teuthology.orchestra.run.smithi110.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2023-01-25T06:29:25.045 INFO:teuthology.orchestra.run.smithi110.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2023-01-25T06:29:25.581 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): ceph-test-18.0.0-1963.g61996d73.el8.x86_ 4.0 MB/s | 47 MB 00:11 2023-01-25T06:29:25.583 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:25.584 INFO:teuthology.orchestra.run.smithi062.stdout:Total 4.1 MB/s | 48 MB 00:11 2023-01-25T06:29:25.628 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:29:25.650 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:29:25.650 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:29:25.971 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:29:25.975 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:29:26.204 INFO:teuthology.orchestra.run.smithi110.stdout:(32/32): ceph-osd-18.0.0-1963.g61996d73.el8.x86 5.1 MB/s | 17 MB 00:03 2023-01-25T06:29:26.206 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:26.207 INFO:teuthology.orchestra.run.smithi110.stdout:Total 6.1 MB/s | 28 MB 00:04 2023-01-25T06:29:26.286 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:29:26.322 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:29:26.323 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:29:26.455 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:29:26.677 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T06:29:26.693 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:29:26.696 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:29:26.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T06:29:26.980 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T06:29:27.054 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T06:29:27.227 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T06:29:27.397 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 5/6 2023-01-25T06:29:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:29:28.157 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T06:29:28.384 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T06:29:28.647 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T06:29:28.933 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T06:29:29.097 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T06:29:29.260 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T06:29:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T06:29:29.607 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T06:29:29.822 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T06:29:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T06:29:30.315 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T06:29:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T06:29:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T06:29:31.171 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T06:29:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T06:29:31.658 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T06:29:31.885 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T06:29:32.242 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T06:29:32.276 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 5/6 2023-01-25T06:29:32.490 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T06:29:32.602 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 6/6 2023-01-25T06:29:32.686 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T06:29:32.897 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T06:29:32.912 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 6/6 2023-01-25T06:29:32.912 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 1/6 2023-01-25T06:29:32.912 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 2/6 2023-01-25T06:29:32.913 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T06:29:32.913 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T06:29:32.913 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T06:29:33.122 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T06:29:33.372 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T06:29:33.468 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T06:29:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:33.469 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:29:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: jq-1.6-6.el8.x86_64 2023-01-25T06:29:33.469 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:33.470 INFO:teuthology.orchestra.run.smithi062.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T06:29:33.470 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T06:29:33.470 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T06:29:33.470 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:33.470 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:29:33.594 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T06:29:33.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph 2023-01-25T06:29:33.936 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T06:29:34.288 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T06:29:34.414 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:12 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8. 27/32 2023-01-25T06:29:34.897 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 28/32 2023-01-25T06:29:35.647 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:29:35.651 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2023-01-25T06:29:35.651 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:29:35.651 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2023-01-25T06:29:35.651 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:29:35.652 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:18.0.0-1963.g61996d73.el8 ceph 6.4 k 2023-01-25T06:29:35.652 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:29:35.652 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:18.0.0-1963.g61996d73.el8 ceph 2.3 M 2023-01-25T06:29:35.652 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:18.0.0-1963.g61996d73.el8 ceph 1.5 M 2023-01-25T06:29:35.653 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 236 k 2023-01-25T06:29:35.653 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.9 M 2023-01-25T06:29:35.653 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 17 M 2023-01-25T06:29:35.653 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T06:29:35.653 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T06:29:35.654 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T06:29:35.654 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T06:29:35.654 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T06:29:35.654 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T06:29:35.654 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T06:29:35.655 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T06:29:35.655 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T06:29:35.655 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T06:29:35.655 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T06:29:35.655 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T06:29:35.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T06:29:35.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T06:29:35.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T06:29:35.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T06:29:35.656 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T06:29:35.657 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T06:29:35.657 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T06:29:35.657 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T06:29:35.657 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T06:29:35.657 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T06:29:35.658 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T06:29:35.658 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T06:29:35.658 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T06:29:35.658 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T06:29:35.658 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:35.659 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:29:35.659 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2023-01-25T06:29:35.659 INFO:teuthology.orchestra.run.smithi062.stdout:Install 32 Packages 2023-01-25T06:29:35.659 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:35.660 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 28 M 2023-01-25T06:29:35.660 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 110 M 2023-01-25T06:29:35.661 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:29:35.927 INFO:teuthology.orchestra.run.smithi062.stdout:(1/32): ceph-18.0.0-1963.g61996d73.el8.x86_64.r 24 kB/s | 6.4 kB 00:00 2023-01-25T06:29:36.439 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 28/32 2023-01-25T06:29:36.540 INFO:teuthology.orchestra.run.smithi062.stdout:(2/32): ceph-mgr-18.0.0-1963.g61996d73.el8.x86_ 1.7 MB/s | 1.5 MB 00:00 2023-01-25T06:29:36.791 INFO:teuthology.orchestra.run.smithi062.stdout:(3/32): ceph-mon-18.0.0-1963.g61996d73.el8.x86_ 4.5 MB/s | 3.9 MB 00:00 2023-01-25T06:29:36.841 INFO:teuthology.orchestra.run.smithi062.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1963.g6199 4.6 MB/s | 236 kB 00:00 2023-01-25T06:29:36.908 INFO:teuthology.orchestra.run.smithi062.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-01-25T06:29:36.950 INFO:teuthology.orchestra.run.smithi062.stdout:(6/32): ceph-mds-18.0.0-1963.g61996d73.el8.x86_ 1.8 MB/s | 2.3 MB 00:01 2023-01-25T06:29:36.977 INFO:teuthology.orchestra.run.smithi062.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 581 kB/s | 39 kB 00:00 2023-01-25T06:29:37.009 INFO:teuthology.orchestra.run.smithi062.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.8 MB/s | 54 kB 00:00 2023-01-25T06:29:37.013 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 29/32 2023-01-25T06:29:37.051 INFO:teuthology.orchestra.run.smithi062.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.0 MB/s | 103 kB 00:00 2023-01-25T06:29:37.084 INFO:teuthology.orchestra.run.smithi062.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 7.0 MB/s | 237 kB 00:00 2023-01-25T06:29:37.118 INFO:teuthology.orchestra.run.smithi062.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.2 MB/s | 457 kB 00:00 2023-01-25T06:29:37.152 INFO:teuthology.orchestra.run.smithi062.stdout:(12/32): python3-cryptography-3.2.1-5.el8.x86_6 8.1 MB/s | 559 kB 00:00 2023-01-25T06:29:37.177 INFO:teuthology.orchestra.run.smithi062.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.8 MB/s | 109 kB 00:00 2023-01-25T06:29:37.353 INFO:teuthology.orchestra.run.smithi062.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 218 kB/s | 44 kB 00:00 2023-01-25T06:29:37.411 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 29/32 2023-01-25T06:29:37.419 INFO:teuthology.orchestra.run.smithi062.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 766 kB/s | 185 kB 00:00 2023-01-25T06:29:37.536 INFO:teuthology.orchestra.run.smithi062.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 940 kB/s | 173 kB 00:00 2023-01-25T06:29:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 30/32 2023-01-25T06:29:37.628 INFO:teuthology.orchestra.run.smithi062.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-01-25T06:29:37.728 INFO:teuthology.orchestra.run.smithi062.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 175 kB/s | 18 kB 00:00 2023-01-25T06:29:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 30/32 2023-01-25T06:29:37.795 INFO:teuthology.orchestra.run.smithi062.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 731 kB/s | 49 kB 00:00 2023-01-25T06:29:37.821 INFO:teuthology.orchestra.run.smithi062.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 957 kB/s | 384 kB 00:00 2023-01-25T06:29:37.959 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 31/32 2023-01-25T06:29:37.963 INFO:teuthology.orchestra.run.smithi062.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 351 kB/s | 59 kB 00:00 2023-01-25T06:29:37.977 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 31/32 2023-01-25T06:29:38.055 INFO:teuthology.orchestra.run.smithi062.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-01-25T06:29:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:18.0.0-1963.g61996d73.el8.x86_64 32/32 2023-01-25T06:29:38.147 INFO:teuthology.orchestra.run.smithi062.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 204 kB/s | 19 kB 00:00 2023-01-25T06:29:38.197 INFO:teuthology.orchestra.run.smithi062.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 752 kB/s | 283 kB 00:00 2023-01-25T06:29:38.214 INFO:teuthology.orchestra.run.smithi062.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 357 kB/s | 24 kB 00:00 2023-01-25T06:29:38.256 INFO:teuthology.orchestra.run.smithi062.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 497 kB/s | 29 kB 00:00 2023-01-25T06:29:38.281 INFO:teuthology.orchestra.run.smithi062.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 410 kB/s | 27 kB 00:00 2023-01-25T06:29:38.474 INFO:teuthology.orchestra.run.smithi062.stdout:(28/32): ceph-osd-18.0.0-1963.g61996d73.el8.x86 8.7 MB/s | 17 MB 00:01 2023-01-25T06:29:38.499 INFO:teuthology.orchestra.run.smithi062.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 995 kB/s | 241 kB 00:00 2023-01-25T06:29:38.506 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-2:18.0.0-1963.g61996d73.el8.x86_64 32/32 2023-01-25T06:29:38.506 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:18.0.0-1963.g61996d73.el8.x86_64 1/32 2023-01-25T06:29:38.507 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 2/32 2023-01-25T06:29:38.507 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 3/32 2023-01-25T06:29:38.507 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 4/32 2023-01-25T06:29:38.507 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 5/32 2023-01-25T06:29:38.507 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8. 6/32 2023-01-25T06:29:38.508 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T06:29:38.508 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T06:29:38.508 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T06:29:38.508 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T06:29:38.508 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T06:29:38.509 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T06:29:38.509 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T06:29:38.509 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T06:29:38.509 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T06:29:38.510 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T06:29:38.510 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T06:29:38.510 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T06:29:38.510 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T06:29:38.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T06:29:38.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T06:29:38.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T06:29:38.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T06:29:38.511 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T06:29:38.512 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T06:29:38.512 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T06:29:38.512 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T06:29:38.512 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T06:29:38.513 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T06:29:38.513 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T06:29:38.513 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T06:29:38.557 INFO:teuthology.orchestra.run.smithi062.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 392 kB/s | 23 kB 00:00 2023-01-25T06:29:38.583 INFO:teuthology.orchestra.run.smithi062.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 834 kB/s | 251 kB 00:00 2023-01-25T06:29:38.641 INFO:teuthology.orchestra.run.smithi062.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 511 kB/s | 85 kB 00:00 2023-01-25T06:29:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:38.645 INFO:teuthology.orchestra.run.smithi062.stdout:Total 9.5 MB/s | 28 MB 00:02 2023-01-25T06:29:38.732 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:29:38.772 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:29:38.772 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:29:39.154 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:29:39.157 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:29:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T06:29:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:39.449 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T06:29:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T06:29:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T06:29:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T06:29:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T06:29:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T06:29:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T06:29:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T06:29:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T06:29:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T06:29:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T06:29:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T06:29:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T06:29:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T06:29:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T06:29:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T06:29:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T06:29:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T06:29:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T06:29:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T06:29:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T06:29:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T06:29:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T06:29:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T06:29:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T06:29:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T06:29:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:39.457 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:39.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-base 2023-01-25T06:29:40.329 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:18 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:40.366 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:29:40.605 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T06:29:40.829 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T06:29:41.088 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T06:29:41.371 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:29:41.394 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T06:29:41.472 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:41.473 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:29:41.473 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:41.555 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install cephadm 2023-01-25T06:29:41.574 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T06:29:41.748 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T06:29:41.940 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T06:29:42.109 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T06:29:42.268 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:20 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:42.336 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T06:29:42.581 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T06:29:43.372 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T06:29:43.405 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:43.406 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:29:43.407 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:43.407 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 82 k 2023-01-25T06:29:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:43.408 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:43.408 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:43.408 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:29:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:43.409 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 82 k 2023-01-25T06:29:43.409 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 374 k 2023-01-25T06:29:43.409 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:29:43.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T06:29:43.870 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm-18.0.0-1963.g61996d73.el8.noarch.rpm 178 kB/s | 82 kB 00:00 2023-01-25T06:29:43.871 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:43.872 INFO:teuthology.orchestra.run.smithi110.stdout:Total 177 kB/s | 82 kB 00:00 2023-01-25T06:29:43.872 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:29:43.879 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:29:43.879 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:29:43.880 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:29:43.884 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:29:43.932 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:29:44.000 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T06:29:44.308 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T06:29:44.503 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T06:29:44.757 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T06:29:44.971 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T06:29:45.286 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T06:29:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:45.485 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T06:29:45.628 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:45.681 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T06:29:45.863 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:45.900 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T06:29:46.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T06:29:46.347 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:46.347 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:46.347 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:46.347 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:46.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:46.348 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:46.411 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T06:29:46.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T06:29:46.660 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T06:29:47.023 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T06:29:47.346 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:25 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:47.389 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T06:29:47.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8. 27/32 2023-01-25T06:29:47.942 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 28/32 2023-01-25T06:29:48.492 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:48.492 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:48.493 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2023-01-25T06:29:48.493 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:48.493 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:48.493 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1963.g61996d73.el8 ceph 165 k 2023-01-25T06:29:48.494 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:48.494 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:48.494 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:48.494 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:29:48.494 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:48.495 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 165 k 2023-01-25T06:29:48.495 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 503 k 2023-01-25T06:29:48.495 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:29:48.894 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-immutable-object-cache-18.0.0-1963.g61996d 414 kB/s | 165 kB 00:00 2023-01-25T06:29:48.895 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:48.895 INFO:teuthology.orchestra.run.smithi110.stdout:Total 412 kB/s | 165 kB 00:00 2023-01-25T06:29:48.896 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:29:48.906 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:29:48.907 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:29:48.955 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:29:48.959 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:29:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:29:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:29:49.550 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 28/32 2023-01-25T06:29:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:29:49.742 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 29/32 2023-01-25T06:29:50.159 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 29/32 2023-01-25T06:29:50.225 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:29:50.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:50.225 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:50.226 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:50.226 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:50.226 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:50.344 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 30/32 2023-01-25T06:29:50.410 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr 2023-01-25T06:29:50.570 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 30/32 2023-01-25T06:29:50.719 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 31/32 2023-01-25T06:29:50.738 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 31/32 2023-01-25T06:29:50.860 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:18.0.0-1963.g61996d73.el8.x86_64 32/32 2023-01-25T06:29:51.138 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:29 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:51.334 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-2:18.0.0-1963.g61996d73.el8.x86_64 32/32 2023-01-25T06:29:51.334 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:18.0.0-1963.g61996d73.el8.x86_64 1/32 2023-01-25T06:29:51.334 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 2/32 2023-01-25T06:29:51.335 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 3/32 2023-01-25T06:29:51.335 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 4/32 2023-01-25T06:29:51.335 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 5/32 2023-01-25T06:29:51.335 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8. 6/32 2023-01-25T06:29:51.335 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T06:29:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T06:29:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T06:29:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T06:29:51.336 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T06:29:51.337 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T06:29:51.338 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T06:29:51.338 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T06:29:51.338 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T06:29:51.338 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T06:29:51.338 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T06:29:51.339 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T06:29:51.339 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T06:29:51.339 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T06:29:51.339 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T06:29:51.339 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T06:29:51.340 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T06:29:51.340 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T06:29:51.340 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T06:29:51.340 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T06:29:52.176 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:29:52.275 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:52.277 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:29:52.277 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:52.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T06:29:52.371 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T06:29:52.371 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:52.371 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:29:52.371 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:52.372 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:52.372 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:52.372 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:52.372 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:52.373 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:29:52.373 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T06:29:52.373 INFO:teuthology.orchestra.run.smithi062.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T06:29:52.373 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T06:29:52.373 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T06:29:52.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T06:29:52.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T06:29:52.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T06:29:52.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T06:29:52.374 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T06:29:52.375 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T06:29:52.375 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T06:29:52.375 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T06:29:52.375 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T06:29:52.376 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T06:29:52.376 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T06:29:52.376 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T06:29:52.376 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T06:29:52.376 INFO:teuthology.orchestra.run.smithi062.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T06:29:52.377 INFO:teuthology.orchestra.run.smithi062.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T06:29:52.377 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T06:29:52.377 INFO:teuthology.orchestra.run.smithi062.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T06:29:52.377 INFO:teuthology.orchestra.run.smithi062.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T06:29:52.377 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T06:29:52.378 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T06:29:52.378 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T06:29:52.378 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T06:29:52.378 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:52.378 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:29:52.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-base 2023-01-25T06:29:53.081 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:31 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:53.288 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:31 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:54.206 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:54.207 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:54.207 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:29:54.207 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:54.208 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:54.208 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 1.7 M 2023-01-25T06:29:54.208 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:29:54.208 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 23 k 2023-01-25T06:29:54.209 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 15 k 2023-01-25T06:29:54.209 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T06:29:54.209 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T06:29:54.209 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T06:29:54.209 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:54.210 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:54.210 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:29:54.210 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2023-01-25T06:29:54.210 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:54.211 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.0 M 2023-01-25T06:29:54.211 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 8.2 M 2023-01-25T06:29:54.211 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:29:54.398 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-base-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:29:54.481 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1963.g619 85 kB/s | 23 kB 00:00 2023-01-25T06:29:54.498 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1963.g6199 50 kB/s | 15 kB 00:00 2023-01-25T06:29:54.510 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:29:54.511 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:29:54.511 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:29:54.548 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 636 kB/s | 43 kB 00:00 2023-01-25T06:29:54.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install cephadm 2023-01-25T06:29:54.715 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 155 kB/s | 34 kB 00:00 2023-01-25T06:29:54.932 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 512 kB/s | 196 kB 00:00 2023-01-25T06:29:55.340 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:55.490 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1963.g61996d73 1.3 MB/s | 1.7 MB 00:01 2023-01-25T06:29:55.493 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:55.493 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2023-01-25T06:29:55.504 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:29:55.513 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:29:55.514 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:29:55.597 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:29:55.600 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:29:55.892 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:29:56.087 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T06:29:56.299 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T06:29:56.492 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T06:29:56.555 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:29:56.556 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:56.556 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:29:56.557 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:56.557 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:29:56.557 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 82 k 2023-01-25T06:29:56.557 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:29:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:29:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:29:56.558 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:29:56.558 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 82 k 2023-01-25T06:29:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 374 k 2023-01-25T06:29:56.559 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:29:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.n 4/6 2023-01-25T06:29:56.933 INFO:teuthology.orchestra.run.smithi062.stdout:cephadm-18.0.0-1963.g61996d73.el8.noarch.rpm 219 kB/s | 82 kB 00:00 2023-01-25T06:29:56.934 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:29:56.934 INFO:teuthology.orchestra.run.smithi062.stdout:Total 218 kB/s | 82 kB 00:00 2023-01-25T06:29:56.934 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:29:56.942 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:29:56.943 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:29:56.945 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:29:56.948 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:29:56.968 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8. 5/6 2023-01-25T06:29:56.997 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:29:57.185 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 6/6 2023-01-25T06:29:57.471 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 6/6 2023-01-25T06:29:57.472 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8. 1/6 2023-01-25T06:29:57.472 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 2/6 2023-01-25T06:29:57.472 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.n 3/6 2023-01-25T06:29:57.472 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T06:29:57.473 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T06:29:57.941 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T06:29:57.941 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:57.942 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:29:57.942 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:57.942 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:57.942 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:29:57.943 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T06:29:57.943 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T06:29:57.943 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T06:29:57.943 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:57.943 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:29:58.050 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T06:29:58.770 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:29:59.331 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:59.483 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:59.733 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:29:59.902 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:29:59.903 INFO:teuthology.orchestra.run.smithi110.stdout:===================================================================================================== 2023-01-25T06:29:59.903 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:29:59.903 INFO:teuthology.orchestra.run.smithi110.stdout:===================================================================================================== 2023-01-25T06:29:59.903 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:29:59.904 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 7.4 M 2023-01-25T06:29:59.904 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:29:59.904 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T06:29:59.904 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T06:29:59.905 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T06:29:59.905 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T06:29:59.905 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T06:29:59.905 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T06:29:59.905 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T06:29:59.906 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:59.906 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:29:59.906 INFO:teuthology.orchestra.run.smithi110.stdout:===================================================================================================== 2023-01-25T06:29:59.906 INFO:teuthology.orchestra.run.smithi110.stdout:Install 8 Packages 2023-01-25T06:29:59.906 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:29:59.907 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 35 M 2023-01-25T06:29:59.907 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 210 M 2023-01-25T06:29:59.907 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:00.130 INFO:teuthology.orchestra.run.smithi110.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-01-25T06:30:00.229 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:00.229 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:00.229 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:00.230 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:00.230 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:00.230 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:00.230 INFO:teuthology.orchestra.run.smithi110.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2023-01-25T06:30:00.263 INFO:teuthology.orchestra.run.smithi110.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-01-25T06:30:00.339 INFO:teuthology.orchestra.run.smithi110.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 18 MB/s | 3.7 MB 00:00 2023-01-25T06:30:00.380 INFO:teuthology.orchestra.run.smithi110.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 15 MB/s | 645 kB 00:00 2023-01-25T06:30:00.414 INFO:teuthology.orchestra.run.smithi110.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-01-25T06:30:00.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T06:30:00.656 INFO:teuthology.orchestra.run.smithi110.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 35 MB/s | 14 MB 00:00 2023-01-25T06:30:01.098 INFO:teuthology.orchestra.run.smithi110.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-196 6.2 MB/s | 7.4 MB 00:01 2023-01-25T06:30:01.100 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:01.101 INFO:teuthology.orchestra.run.smithi110.stdout:Total 30 MB/s | 35 MB 00:01 2023-01-25T06:30:01.181 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:39 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:01.332 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:01.350 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:01.350 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:01.746 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:01.749 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:02.407 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:02.409 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:02.409 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-25T06:30:02.409 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:02.409 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:02.409 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1963.g61996d73.el8 ceph 165 k 2023-01-25T06:30:02.410 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:02.410 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:02.410 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:02.410 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:30:02.411 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:02.411 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 165 k 2023-01-25T06:30:02.411 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 503 k 2023-01-25T06:30:02.411 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:02.472 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T06:30:02.635 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-immutable-object-cache-18.0.0-1963.g61996d 734 kB/s | 165 kB 00:00 2023-01-25T06:30:02.636 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:02.636 INFO:teuthology.orchestra.run.smithi062.stdout:Total 730 kB/s | 165 kB 00:00 2023-01-25T06:30:02.636 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T06:30:02.648 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:02.648 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:02.700 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:02.704 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:02.764 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T06:30:02.850 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:02.993 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:30:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T06:30:03.437 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:30:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:03.947 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1963.g61996d73. 1/1 2023-01-25T06:30:03.947 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:03.947 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:03.948 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:03.948 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:03.948 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:04.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr 2023-01-25T06:30:04.167 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:04.697 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T06:30:04.878 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:42 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:05.381 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T06:30:05.717 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T06:30:05.947 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T06:30:05.991 INFO:teuthology.orchestra.run.smithi062.stdout:Package ceph-mgr-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:06.103 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:06.105 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:30:06.105 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:06.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T06:30:06.942 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:07.721 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T06:30:08.178 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:08.179 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:08.179 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:08.180 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:08.180 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:08.180 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 1.7 M 2023-01-25T06:30:08.180 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:30:08.180 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 23 k 2023-01-25T06:30:08.181 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 15 k 2023-01-25T06:30:08.181 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T06:30:08.181 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T06:30:08.181 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T06:30:08.181 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:08.182 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:08.182 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:08.182 INFO:teuthology.orchestra.run.smithi062.stdout:Install 6 Packages 2023-01-25T06:30:08.182 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:08.183 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 2.0 M 2023-01-25T06:30:08.183 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 8.2 M 2023-01-25T06:30:08.183 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:08.441 INFO:teuthology.orchestra.run.smithi062.stdout:(1/6): ceph-prometheus-alerts-18.0.0-1963.g6199 56 kB/s | 15 kB 00:00 2023-01-25T06:30:08.467 INFO:teuthology.orchestra.run.smithi062.stdout:(2/6): ceph-grafana-dashboards-18.0.0-1963.g619 81 kB/s | 23 kB 00:00 2023-01-25T06:30:08.809 INFO:teuthology.orchestra.run.smithi062.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 98 kB/s | 34 kB 00:00 2023-01-25T06:30:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T06:30:08.992 INFO:teuthology.orchestra.run.smithi062.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-01-25T06:30:09.180 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 8/8 2023-01-25T06:30:09.326 INFO:teuthology.orchestra.run.smithi062.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1963.g61996d73 1.5 MB/s | 1.7 MB 00:01 2023-01-25T06:30:09.516 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 8/8 2023-01-25T06:30:09.516 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 1/8 2023-01-25T06:30:09.517 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T06:30:09.517 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:09.517 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T06:30:09.517 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T06:30:09.517 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T06:30:09.518 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T06:30:10.570 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T06:30:10.571 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:10.571 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:10.571 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:10.571 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T06:30:10.572 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T06:30:10.572 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T06:30:10.572 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T06:30:10.572 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T06:30:10.572 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T06:30:10.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T06:30:10.573 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:10.573 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:10.684 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-rook 2023-01-25T06:30:11.407 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:11.884 INFO:teuthology.orchestra.run.smithi062.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 12 kB/s | 43 kB 00:03 2023-01-25T06:30:11.887 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:11.887 INFO:teuthology.orchestra.run.smithi062.stdout:Total 559 kB/s | 2.0 MB 00:03 2023-01-25T06:30:11.898 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:11.908 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:11.909 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:12.002 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:12.006 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:12.296 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:12.516 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T06:30:12.539 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:12.541 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================== 2023-01-25T06:30:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:30:12.541 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================== 2023-01-25T06:30:12.541 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 47 k 2023-01-25T06:30:12.542 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:30:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T06:30:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T06:30:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T06:30:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T06:30:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T06:30:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T06:30:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T06:30:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T06:30:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T06:30:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T06:30:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T06:30:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T06:30:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:12.545 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:12.545 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================== 2023-01-25T06:30:12.545 INFO:teuthology.orchestra.run.smithi110.stdout:Install 13 Packages 2023-01-25T06:30:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:12.546 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 1.9 M 2023-01-25T06:30:12.546 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 25 M 2023-01-25T06:30:12.547 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:12.614 INFO:teuthology.orchestra.run.smithi110.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 387 kB/s | 27 kB 00:00 2023-01-25T06:30:12.630 INFO:teuthology.orchestra.run.smithi110.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 233 kB/s | 20 kB 00:00 2023-01-25T06:30:12.656 INFO:teuthology.orchestra.run.smithi110.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-01-25T06:30:12.672 INFO:teuthology.orchestra.run.smithi110.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.6 MB/s | 110 kB 00:00 2023-01-25T06:30:12.698 INFO:teuthology.orchestra.run.smithi110.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.6 MB/s | 155 kB 00:00 2023-01-25T06:30:12.709 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T06:30:12.723 INFO:teuthology.orchestra.run.smithi110.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 846 kB/s | 43 kB 00:00 2023-01-25T06:30:12.840 INFO:teuthology.orchestra.run.smithi110.stdout:(7/13): ceph-mgr-rook-18.0.0-1963.g61996d73.el8 160 kB/s | 47 kB 00:00 2023-01-25T06:30:12.893 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T06:30:12.991 INFO:teuthology.orchestra.run.smithi110.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 113 kB/s | 33 kB 00:00 2023-01-25T06:30:13.006 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.n 4/6 2023-01-25T06:30:13.091 INFO:teuthology.orchestra.run.smithi110.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 324 kB/s | 81 kB 00:00 2023-01-25T06:30:13.233 INFO:teuthology.orchestra.run.smithi110.stdout:(10/13): python3-rsa-4.8-1.el8.noarch.rpm 434 kB/s | 61 kB 00:00 2023-01-25T06:30:13.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8. 5/6 2023-01-25T06:30:13.383 INFO:teuthology.orchestra.run.smithi110.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 408 kB/s | 61 kB 00:00 2023-01-25T06:30:13.450 INFO:teuthology.orchestra.run.smithi110.stdout:(12/13): python3-certifi-2018.10.15-7.el8.noarc 22 kB/s | 16 kB 00:00 2023-01-25T06:30:13.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 6/6 2023-01-25T06:30:13.875 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 6/6 2023-01-25T06:30:13.876 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8. 1/6 2023-01-25T06:30:13.876 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarc 2/6 2023-01-25T06:30:13.876 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.n 3/6 2023-01-25T06:30:13.876 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T06:30:13.877 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T06:30:14.393 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T06:30:14.393 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:14.393 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:14.394 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:14.394 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:14.394 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:14.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T06:30:14.394 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T06:30:14.395 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T06:30:14.395 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:14.395 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:14.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T06:30:14.633 INFO:teuthology.orchestra.run.smithi110.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 727 kB/s | 1.2 MB 00:01 2023-01-25T06:30:14.635 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:14.636 INFO:teuthology.orchestra.run.smithi110.stdout:Total 943 kB/s | 1.9 MB 00:02 2023-01-25T06:30:14.692 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:14.707 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:14.707 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:14.806 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:14.810 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:15.253 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:01:53 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:15.324 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T06:30:15.755 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T06:30:16.349 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T06:30:16.488 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:16.489 INFO:teuthology.orchestra.run.smithi062.stdout:===================================================================================================== 2023-01-25T06:30:16.489 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:16.490 INFO:teuthology.orchestra.run.smithi062.stdout:===================================================================================================== 2023-01-25T06:30:16.490 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 7.4 M 2023-01-25T06:30:16.490 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:30:16.490 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T06:30:16.491 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T06:30:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:16.492 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:16.492 INFO:teuthology.orchestra.run.smithi062.stdout:===================================================================================================== 2023-01-25T06:30:16.492 INFO:teuthology.orchestra.run.smithi062.stdout:Install 8 Packages 2023-01-25T06:30:16.492 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:16.493 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 35 M 2023-01-25T06:30:16.493 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 210 M 2023-01-25T06:30:16.494 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T06:30:16.727 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T06:30:16.758 INFO:teuthology.orchestra.run.smithi062.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2023-01-25T06:30:16.858 INFO:teuthology.orchestra.run.smithi062.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-01-25T06:30:16.892 INFO:teuthology.orchestra.run.smithi062.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-01-25T06:30:16.941 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T06:30:16.975 INFO:teuthology.orchestra.run.smithi062.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2023-01-25T06:30:17.018 INFO:teuthology.orchestra.run.smithi062.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 15 MB/s | 645 kB 00:00 2023-01-25T06:30:17.051 INFO:teuthology.orchestra.run.smithi062.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-01-25T06:30:17.268 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T06:30:17.327 INFO:teuthology.orchestra.run.smithi062.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 32 MB/s | 14 MB 00:00 2023-01-25T06:30:17.516 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T06:30:17.652 INFO:teuthology.orchestra.run.smithi062.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-196 6.4 MB/s | 7.4 MB 00:01 2023-01-25T06:30:17.654 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:17.655 INFO:teuthology.orchestra.run.smithi062.stdout:Total 30 MB/s | 35 MB 00:01 2023-01-25T06:30:17.892 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:17.913 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:17.913 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:18.332 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:18.336 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:18.550 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T06:30:18.810 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T06:30:18.921 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:19.002 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T06:30:19.065 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T06:30:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T06:30:19.235 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T06:30:19.345 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 13/13 2023-01-25T06:30:19.382 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T06:30:19.639 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 13/13 2023-01-25T06:30:19.639 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 1/13 2023-01-25T06:30:19.639 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T06:30:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T06:30:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T06:30:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T06:30:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T06:30:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T06:30:19.641 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T06:30:19.641 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T06:30:19.641 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T06:30:19.641 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T06:30:19.641 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T06:30:20.018 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T06:30:20.184 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T06:30:20.185 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:20.185 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:20.185 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:20.185 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T06:30:20.185 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T06:30:20.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T06:30:20.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T06:30:20.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T06:30:20.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T06:30:20.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T06:30:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T06:30:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T06:30:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T06:30:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T06:30:20.187 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T06:30:20.188 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:20.188 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:20.215 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:20.307 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T06:30:20.876 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:21.030 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:59 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:21.031 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T06:30:21.782 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T06:30:22.179 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:22.180 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:22.181 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:30:22.181 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:22.181 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:22.181 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 120 k 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T06:30:22.182 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:22.183 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:22.183 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:22.183 INFO:teuthology.orchestra.run.smithi110.stdout:Install 5 Packages 2023-01-25T06:30:22.183 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:22.184 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 6.0 M 2023-01-25T06:30:22.184 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 26 M 2023-01-25T06:30:22.185 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:22.265 INFO:teuthology.orchestra.run.smithi110.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 6.4 MB/s | 538 kB 00:00 2023-01-25T06:30:22.365 INFO:teuthology.orchestra.run.smithi110.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-01-25T06:30:22.416 INFO:teuthology.orchestra.run.smithi110.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1963.g61996d73.e 516 kB/s | 120 kB 00:00 2023-01-25T06:30:22.549 INFO:teuthology.orchestra.run.smithi110.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 328 kB/s | 60 kB 00:00 2023-01-25T06:30:22.600 INFO:teuthology.orchestra.run.smithi110.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2023-01-25T06:30:22.601 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:22.602 INFO:teuthology.orchestra.run.smithi110.stdout:Total 14 MB/s | 6.0 MB 00:00 2023-01-25T06:30:22.627 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T06:30:22.652 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:22.663 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:22.664 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:22.792 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:22.795 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:22.856 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T06:30:23.077 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T06:30:24.345 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T06:30:24.742 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T06:30:24.773 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T06:30:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T06:30:25.236 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 5/5 2023-01-25T06:30:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 5/5 2023-01-25T06:30:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/5 2023-01-25T06:30:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T06:30:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T06:30:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T06:30:26.016 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T06:30:26.033 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T06:30:26.033 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:26.033 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:26.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:26.034 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T06:30:26.034 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T06:30:26.034 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T06:30:26.034 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T06:30:26.034 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:26.035 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:26.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-fuse 2023-01-25T06:30:26.214 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 8/8 2023-01-25T06:30:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 8/8 2023-01-25T06:30:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d7 1/8 2023-01-25T06:30:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T06:30:26.559 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T06:30:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T06:30:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T06:30:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T06:30:26.560 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T06:30:26.863 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T06:30:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:27.095 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:27.095 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T06:30:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T06:30:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T06:30:27.096 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T06:30:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T06:30:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T06:30:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T06:30:27.097 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:27.097 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:27.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-rook 2023-01-25T06:30:27.968 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:05 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:28.010 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:28.010 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:28.011 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:30:28.011 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:28.011 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:28.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:18.0.0-1963.g61996d73.el8 ceph 845 k 2023-01-25T06:30:28.012 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-01-25T06:30:28.012 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T06:30:28.012 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:28.012 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:28.012 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:28.013 INFO:teuthology.orchestra.run.smithi110.stdout:Install 2 Packages 2023-01-25T06:30:28.013 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:28.013 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 928 k 2023-01-25T06:30:28.014 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 2.9 M 2023-01-25T06:30:28.014 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:28.139 INFO:teuthology.orchestra.run.smithi110.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 656 kB/s | 83 kB 00:00 2023-01-25T06:30:28.348 INFO:teuthology.orchestra.run.smithi110.stdout:(2/2): ceph-fuse-18.0.0-1963.g61996d73.el8.x86_ 2.5 MB/s | 845 kB 00:00 2023-01-25T06:30:28.350 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:28.350 INFO:teuthology.orchestra.run.smithi110.stdout:Total 2.7 MB/s | 928 kB 00:00 2023-01-25T06:30:28.360 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:28.380 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:28.381 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:28.476 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:28.479 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:28.664 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:28.917 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T06:30:29.132 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2/2 2023-01-25T06:30:29.201 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:29.203 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================== 2023-01-25T06:30:29.203 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:29.203 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================== 2023-01-25T06:30:29.204 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:29.204 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 47 k 2023-01-25T06:30:29.204 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:30:29.204 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T06:30:29.205 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T06:30:29.206 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T06:30:29.206 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T06:30:29.206 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T06:30:29.206 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T06:30:29.206 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T06:30:29.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================== 2023-01-25T06:30:29.207 INFO:teuthology.orchestra.run.smithi062.stdout:Install 13 Packages 2023-01-25T06:30:29.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 1.9 M 2023-01-25T06:30:29.208 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 25 M 2023-01-25T06:30:29.209 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:29.269 INFO:teuthology.orchestra.run.smithi062.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 430 kB/s | 27 kB 00:00 2023-01-25T06:30:29.302 INFO:teuthology.orchestra.run.smithi062.stdout:(2/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.7 MB/s | 126 kB 00:00 2023-01-25T06:30:29.328 INFO:teuthology.orchestra.run.smithi062.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 165 kB/s | 20 kB 00:00 2023-01-25T06:30:29.345 INFO:teuthology.orchestra.run.smithi062.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.6 MB/s | 110 kB 00:00 2023-01-25T06:30:29.387 INFO:teuthology.orchestra.run.smithi062.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.6 MB/s | 155 kB 00:00 2023-01-25T06:30:29.412 INFO:teuthology.orchestra.run.smithi062.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 635 kB/s | 43 kB 00:00 2023-01-25T06:30:29.539 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2/2 2023-01-25T06:30:29.539 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/2 2023-01-25T06:30:29.579 INFO:teuthology.orchestra.run.smithi062.stdout:(7/13): ceph-mgr-rook-18.0.0-1963.g61996d73.el8 127 kB/s | 47 kB 00:00 2023-01-25T06:30:29.621 INFO:teuthology.orchestra.run.smithi062.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 78 kB/s | 16 kB 00:00 2023-01-25T06:30:29.647 INFO:teuthology.orchestra.run.smithi062.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 127 kB/s | 33 kB 00:00 2023-01-25T06:30:29.764 INFO:teuthology.orchestra.run.smithi062.stdout:(10/13): python3-rsa-4.8-1.el8.noarch.rpm 527 kB/s | 61 kB 00:00 2023-01-25T06:30:29.797 INFO:teuthology.orchestra.run.smithi062.stdout:(11/13): python3-google-auth-1.1.1-10.el8.noarc 373 kB/s | 81 kB 00:00 2023-01-25T06:30:29.906 INFO:teuthology.orchestra.run.smithi062.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 431 kB/s | 61 kB 00:00 2023-01-25T06:30:30.019 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T06:30:30.019 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:30.020 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:30.020 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T06:30:30.021 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:30.021 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:30.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-volume 2023-01-25T06:30:30.339 INFO:teuthology.orchestra.run.smithi062.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.6 MB/s | 1.2 MB 00:00 2023-01-25T06:30:30.342 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:30.342 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.7 MB/s | 1.9 MB 00:01 2023-01-25T06:30:30.405 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:30.421 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:30.422 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:30.529 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:30.533 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:30.942 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:31.077 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:31.336 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T06:30:31.525 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T06:30:31.759 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T06:30:32.004 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T06:30:32.081 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:32.082 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:32.082 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-01-25T06:30:32.082 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:32.083 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:32.083 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 263 k 2023-01-25T06:30:32.083 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:32.083 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:32.083 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:32.084 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:30:32.084 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:32.084 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 263 k 2023-01-25T06:30:32.084 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 1.3 M 2023-01-25T06:30:32.084 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:32.170 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T06:30:32.319 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-volume-18.0.0-1963.g61996d73.el8.noarch.rp 1.1 MB/s | 263 kB 00:00 2023-01-25T06:30:32.319 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:32.320 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.1 MB/s | 263 kB 00:00 2023-01-25T06:30:32.320 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:32.330 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:32.330 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:32.387 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T06:30:32.414 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:32.417 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:32.599 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:32.682 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T06:30:32.877 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:33.155 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T06:30:33.335 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:33.809 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:33.810 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:33.810 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:33.810 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:33.810 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:33.811 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:34.047 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados-devel 2023-01-25T06:30:34.261 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T06:30:34.504 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T06:30:34.688 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T06:30:34.776 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:12 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:34.881 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T06:30:35.054 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 13/13 2023-01-25T06:30:35.377 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 13/13 2023-01-25T06:30:35.377 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 1/13 2023-01-25T06:30:35.377 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T06:30:35.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T06:30:35.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T06:30:35.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T06:30:35.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T06:30:35.378 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T06:30:35.379 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T06:30:35.379 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T06:30:35.379 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T06:30:35.380 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T06:30:35.380 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T06:30:35.923 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:35.924 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2023-01-25T06:30:35.924 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:35.924 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:18.0.0-1963.g61996d73.el8 ceph 139 k 2023-01-25T06:30:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:35.925 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:35.925 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:35.925 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:30:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:35.926 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 139 k 2023-01-25T06:30:35.926 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 512 k 2023-01-25T06:30:35.926 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:35.954 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T06:30:35.955 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:35.955 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:35.955 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:35.955 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T06:30:35.956 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T06:30:35.956 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T06:30:35.956 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T06:30:35.956 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T06:30:35.956 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T06:30:35.957 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T06:30:35.957 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T06:30:35.957 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T06:30:35.957 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T06:30:35.958 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T06:30:35.958 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T06:30:35.958 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:35.958 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:36.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T06:30:36.130 INFO:teuthology.orchestra.run.smithi110.stdout:librados-devel-18.0.0-1963.g61996d73.el8.x86_64 675 kB/s | 139 kB 00:00 2023-01-25T06:30:36.131 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:36.131 INFO:teuthology.orchestra.run.smithi110.stdout:Total 671 kB/s | 139 kB 00:00 2023-01-25T06:30:36.132 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:36.136 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:36.137 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:36.189 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:36.193 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:36.324 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:36.507 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:36.801 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:36.838 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:14 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:37.730 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:37.731 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:38.080 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:38.080 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:38.081 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:38.081 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:38.081 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:38.081 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 120 k 2023-01-25T06:30:38.082 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:30:38.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T06:30:38.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T06:30:38.082 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T06:30:38.083 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T06:30:38.083 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:38.083 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:38.083 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:38.083 INFO:teuthology.orchestra.run.smithi062.stdout:Install 5 Packages 2023-01-25T06:30:38.084 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:38.084 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 6.0 M 2023-01-25T06:30:38.084 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 26 M 2023-01-25T06:30:38.085 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:38.164 INFO:teuthology.orchestra.run.smithi062.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 6.5 MB/s | 538 kB 00:00 2023-01-25T06:30:38.281 INFO:teuthology.orchestra.run.smithi062.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-01-25T06:30:38.351 INFO:teuthology.orchestra.run.smithi062.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1963.g61996d73.e 539 kB/s | 120 kB 00:00 2023-01-25T06:30:38.440 INFO:teuthology.orchestra.run.smithi062.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 380 kB/s | 60 kB 00:00 2023-01-25T06:30:38.490 INFO:teuthology.orchestra.run.smithi062.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2023-01-25T06:30:38.492 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:38.492 INFO:teuthology.orchestra.run.smithi062.stdout:Total 15 MB/s | 6.0 MB 00:00 2023-01-25T06:30:38.545 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:38.556 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:38.557 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:38.702 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:38.705 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:38.994 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:39.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T06:30:39.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs2 2023-01-25T06:30:40.270 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T06:30:40.314 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:18 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:40.674 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T06:30:40.942 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T06:30:41.120 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 5/5 2023-01-25T06:30:41.356 INFO:teuthology.orchestra.run.smithi110.stdout:Package libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 5/5 2023-01-25T06:30:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 1/5 2023-01-25T06:30:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T06:30:41.446 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T06:30:41.447 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T06:30:41.456 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:41.457 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:41.457 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:41.541 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs-devel 2023-01-25T06:30:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T06:30:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:41.970 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:41.970 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T06:30:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T06:30:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T06:30:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T06:30:41.971 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:41.972 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:42.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-fuse 2023-01-25T06:30:42.254 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:42.848 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:43.402 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:43.403 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2023-01-25T06:30:43.404 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:43.404 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:18.0.0-1963.g61996d73.el8 ceph 30 k 2023-01-25T06:30:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:43.405 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:43.405 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:43.405 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:30:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:43.405 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 30 k 2023-01-25T06:30:43.406 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 137 k 2023-01-25T06:30:43.406 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:43.561 INFO:teuthology.orchestra.run.smithi110.stdout:libcephfs-devel-18.0.0-1963.g61996d73.el8.x86_6 194 kB/s | 30 kB 00:00 2023-01-25T06:30:43.562 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:43.562 INFO:teuthology.orchestra.run.smithi110.stdout:Total 193 kB/s | 30 kB 00:00 2023-01-25T06:30:43.562 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:43.565 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:43.566 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:43.588 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:43.592 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:43.653 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:44.067 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:44.069 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:44.069 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:44.069 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:44.069 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:44.070 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:18.0.0-1963.g61996d73.el8 ceph 845 k 2023-01-25T06:30:44.070 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2023-01-25T06:30:44.070 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T06:30:44.070 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:44.070 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:44.071 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:44.071 INFO:teuthology.orchestra.run.smithi062.stdout:Install 2 Packages 2023-01-25T06:30:44.071 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:44.071 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 928 k 2023-01-25T06:30:44.071 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 2.9 M 2023-01-25T06:30:44.072 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:44.133 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:44.140 INFO:teuthology.orchestra.run.smithi062.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-01-25T06:30:44.407 INFO:teuthology.orchestra.run.smithi062.stdout:(2/2): ceph-fuse-18.0.0-1963.g61996d73.el8.x86_ 2.5 MB/s | 845 kB 00:00 2023-01-25T06:30:44.407 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:44.408 INFO:teuthology.orchestra.run.smithi062.stdout:Total 2.7 MB/s | 928 kB 00:00 2023-01-25T06:30:44.418 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:44.441 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:44.441 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:44.541 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:44.545 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:44.571 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:44.572 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:44.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados2 2023-01-25T06:30:44.736 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:45.023 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T06:30:45.216 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2/2 2023-01-25T06:30:45.398 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:45.642 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2/2 2023-01-25T06:30:45.643 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/2 2023-01-25T06:30:46.161 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T06:30:46.161 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:46.162 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:46.162 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T06:30:46.162 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:46.162 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:46.345 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-volume 2023-01-25T06:30:46.437 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:46.539 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:46.540 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:46.540 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:46.649 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librbd1 2023-01-25T06:30:47.111 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:47.375 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:48.331 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:48.332 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:48.332 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2023-01-25T06:30:48.332 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:48.332 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:48.333 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:18.0.0-1963.g61996d73.el8 ceph-noarch 263 k 2023-01-25T06:30:48.333 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:48.333 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:48.333 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:48.333 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:30:48.334 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:48.334 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 263 k 2023-01-25T06:30:48.334 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 1.3 M 2023-01-25T06:30:48.334 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:48.410 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:48.512 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:48.513 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:48.513 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:48.553 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-volume-18.0.0-1963.g61996d73.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-01-25T06:30:48.554 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:48.554 INFO:teuthology.orchestra.run.smithi062.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-01-25T06:30:48.554 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:48.565 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:48.565 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:48.604 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rados 2023-01-25T06:30:48.659 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:48.663 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:48.882 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:49.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:49.340 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:49.598 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:50.110 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 1/1 2023-01-25T06:30:50.110 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:50.110 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:50.110 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:18.0.0-1963.g61996d73.el8.noarch 2023-01-25T06:30:50.111 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:50.111 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:50.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados-devel 2023-01-25T06:30:50.389 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:50.491 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:50.492 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:50.492 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:50.588 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rgw 2023-01-25T06:30:51.060 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:51.306 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:52.278 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:52.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:52.279 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-25T06:30:52.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:52.279 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:52.280 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:18.0.0-1963.g61996d73.el8 ceph 139 k 2023-01-25T06:30:52.280 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:52.280 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:52.280 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:52.280 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:30:52.281 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:52.281 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 139 k 2023-01-25T06:30:52.281 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 512 k 2023-01-25T06:30:52.281 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:52.339 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:52.440 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:52.441 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:52.441 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:52.474 INFO:teuthology.orchestra.run.smithi062.stdout:librados-devel-18.0.0-1963.g61996d73.el8.x86_64 717 kB/s | 139 kB 00:00 2023-01-25T06:30:52.475 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:52.475 INFO:teuthology.orchestra.run.smithi062.stdout:Total 712 kB/s | 139 kB 00:00 2023-01-25T06:30:52.475 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:52.480 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:52.480 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:52.535 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:52.537 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-cephfs 2023-01-25T06:30:52.539 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:52.658 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:52.818 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:53.124 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:53.256 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:53.645 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:53.646 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:53.646 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:53.646 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:53.646 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:53.647 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:54.301 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:54.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs2 2023-01-25T06:30:54.401 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:54.402 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:54.402 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:54.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rbd 2023-01-25T06:30:55.141 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:55.218 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:56.255 INFO:teuthology.orchestra.run.smithi062.stdout:Package libcephfs2-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:56.264 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:30:56.366 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2023-01-25T06:30:56.367 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:56.368 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:56.369 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:30:56.369 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:56.458 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-fuse 2023-01-25T06:30:56.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install libcephfs-devel 2023-01-25T06:30:57.197 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:35 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:57.228 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:35 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:30:58.344 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:30:58.345 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:58.345 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2023-01-25T06:30:58.346 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:58.346 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:30:58.346 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:18.0.0-1963.g61996d73.el8 ceph 94 k 2023-01-25T06:30:58.346 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:58.346 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:30:58.347 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:30:58.347 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:30:58.347 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:58.347 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 94 k 2023-01-25T06:30:58.347 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 280 k 2023-01-25T06:30:58.348 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:30:58.459 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:30:58.459 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:58.460 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-25T06:30:58.460 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:58.460 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:30:58.460 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:18.0.0-1963.g61996d73.el8 ceph 30 k 2023-01-25T06:30:58.461 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:58.461 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:30:58.461 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:30:58.461 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:30:58.461 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:58.462 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 30 k 2023-01-25T06:30:58.462 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 137 k 2023-01-25T06:30:58.462 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:30:58.526 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-fuse-18.0.0-1963.g61996d73.el8.x86_64.rpm 523 kB/s | 94 kB 00:00 2023-01-25T06:30:58.527 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:58.527 INFO:teuthology.orchestra.run.smithi110.stdout:Total 520 kB/s | 94 kB 00:00 2023-01-25T06:30:58.528 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:30:58.537 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:30:58.538 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:30:58.590 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:30:58.593 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:30:58.640 INFO:teuthology.orchestra.run.smithi062.stdout:libcephfs-devel-18.0.0-1963.g61996d73.el8.x86_6 171 kB/s | 30 kB 00:00 2023-01-25T06:30:58.640 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:30:58.641 INFO:teuthology.orchestra.run.smithi062.stdout:Total 169 kB/s | 30 kB 00:00 2023-01-25T06:30:58.641 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:30:58.644 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:30:58.645 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:30:58.672 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:30:58.676 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:30:58.699 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:30:58.744 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:30:58.920 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:58.942 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:59.216 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:59.225 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:59.687 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:59.687 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:59.687 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:30:59.688 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:59.688 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:30:59.688 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:30:59.702 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:30:59.703 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:59.703 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:30:59.703 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:30:59.703 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:30:59.704 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:30:59.801 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librados2 2023-01-25T06:30:59.901 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-mirror 2023-01-25T06:31:00.567 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:00.629 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:01.676 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:01.766 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:31:01.767 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:01.768 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2023-01-25T06:31:01.768 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:01.768 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:31:01.768 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.4 M 2023-01-25T06:31:01.769 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:01.769 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:31:01.769 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:01.769 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:31:01.769 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:01.770 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.4 M 2023-01-25T06:31:01.770 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 16 M 2023-01-25T06:31:01.770 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:31:01.787 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:01.788 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:01.789 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:01.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install librbd1 2023-01-25T06:31:02.389 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-mirror-18.0.0-1963.g61996d73.el8.x86_64.rpm 5.6 MB/s | 3.4 MB 00:00 2023-01-25T06:31:02.390 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:31:02.390 INFO:teuthology.orchestra.run.smithi110.stdout:Total 5.5 MB/s | 3.4 MB 00:00 2023-01-25T06:31:02.391 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:31:02.402 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:31:02.402 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:31:02.466 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:31:02.470 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:31:02.642 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:40 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:31:03.106 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:03.776 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:03.891 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:03.892 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:03.892 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:03.989 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rados 2023-01-25T06:31:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:31:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:31:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:04.048 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:31:04.255 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-nbd 2023-01-25T06:31:04.755 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:04.996 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:05.864 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rados-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:05.976 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:05.978 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:05.978 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:06.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rgw 2023-01-25T06:31:06.141 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-01-25T06:31:06.142 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2023-01-25T06:31:06.142 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:06.143 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-01-25T06:31:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 179 k 2023-01-25T06:31:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:06.143 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-01-25T06:31:06.143 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-01-25T06:31:06.144 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2023-01-25T06:31:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:06.144 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 179 k 2023-01-25T06:31:06.144 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 568 k 2023-01-25T06:31:06.144 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-01-25T06:31:06.348 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-nbd-18.0.0-1963.g61996d73.el8.x86_64.rpm 874 kB/s | 179 kB 00:00 2023-01-25T06:31:06.349 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:31:06.349 INFO:teuthology.orchestra.run.smithi110.stdout:Total 870 kB/s | 179 kB 00:00 2023-01-25T06:31:06.349 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-01-25T06:31:06.360 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-01-25T06:31:06.360 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-01-25T06:31:06.411 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-01-25T06:31:06.415 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-01-25T06:31:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-01-25T06:31:06.720 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:06.824 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:44 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:07.021 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:07.484 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:07.484 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:07.485 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-01-25T06:31:07.485 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:31:07.485 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:07.485 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-01-25T06:31:07.692 DEBUG:teuthology.parallel:result is None 2023-01-25T06:31:07.935 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rgw-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:08.047 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:08.049 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:08.049 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:08.136 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-cephfs 2023-01-25T06:31:08.894 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:46 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:09.998 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-cephfs-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:10.111 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:10.113 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:10.113 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:10.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install python3-rbd 2023-01-25T06:31:10.975 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:48 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:12.084 INFO:teuthology.orchestra.run.smithi062.stdout:Package python3-rbd-2:18.0.0-1963.g61996d73.el8.x86_64 is already installed. 2023-01-25T06:31:12.198 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:12.200 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2023-01-25T06:31:12.200 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:12.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-fuse 2023-01-25T06:31:13.040 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:51 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:14.278 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:14.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:14.279 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-01-25T06:31:14.279 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:14.279 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:31:14.280 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:18.0.0-1963.g61996d73.el8 ceph 94 k 2023-01-25T06:31:14.280 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:14.280 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:31:14.280 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:14.281 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:31:14.281 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:14.281 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 94 k 2023-01-25T06:31:14.281 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 280 k 2023-01-25T06:31:14.281 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:31:14.480 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-fuse-18.0.0-1963.g61996d73.el8.x86_64.rpm 472 kB/s | 94 kB 00:00 2023-01-25T06:31:14.481 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:31:14.481 INFO:teuthology.orchestra.run.smithi062.stdout:Total 469 kB/s | 94 kB 00:00 2023-01-25T06:31:14.481 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:31:14.492 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:31:14.493 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:31:14.547 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:31:14.551 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:31:14.667 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:31:14.857 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:15.190 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:15.682 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:15.682 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:15.682 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:31:15.682 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:31:15.683 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:15.683 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:15.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-mirror 2023-01-25T06:31:16.702 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:54 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:17.932 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:17.933 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:17.933 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repo Size 2023-01-25T06:31:17.933 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:17.933 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:31:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:18.0.0-1963.g61996d73.el8 ceph 3.4 M 2023-01-25T06:31:17.934 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:17.934 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:31:17.934 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:17.934 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:31:17.935 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:17.935 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 3.4 M 2023-01-25T06:31:17.935 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 16 M 2023-01-25T06:31:17.935 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:31:18.482 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-mirror-18.0.0-1963.g61996d73.el8.x86_64.rpm 6.3 MB/s | 3.4 MB 00:00 2023-01-25T06:31:18.483 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:31:18.483 INFO:teuthology.orchestra.run.smithi062.stdout:Total 6.3 MB/s | 3.4 MB 00:00 2023-01-25T06:31:18.484 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:31:18.497 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:31:18.498 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:31:18.566 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:31:18.569 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:31:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:31:19.199 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:19.670 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:20.205 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:20.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:20.205 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:31:20.206 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:31:20.206 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:20.206 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:20.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install rbd-nbd 2023-01-25T06:31:21.153 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:02:59 ago on Wed 25 Jan 2023 06:28:22 AM UTC. 2023-01-25T06:31:22.377 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2023-01-25T06:31:22.378 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:22.378 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2023-01-25T06:31:22.378 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:22.379 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2023-01-25T06:31:22.379 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:18.0.0-1963.g61996d73.el8 ceph 179 k 2023-01-25T06:31:22.379 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:22.379 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2023-01-25T06:31:22.379 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2023-01-25T06:31:22.380 INFO:teuthology.orchestra.run.smithi062.stdout:Install 1 Package 2023-01-25T06:31:22.380 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:22.380 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 179 k 2023-01-25T06:31:22.381 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 568 k 2023-01-25T06:31:22.381 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2023-01-25T06:31:22.761 INFO:teuthology.orchestra.run.smithi062.stdout:rbd-nbd-18.0.0-1963.g61996d73.el8.x86_64.rpm 470 kB/s | 179 kB 00:00 2023-01-25T06:31:22.762 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2023-01-25T06:31:22.762 INFO:teuthology.orchestra.run.smithi062.stdout:Total 469 kB/s | 179 kB 00:00 2023-01-25T06:31:22.762 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2023-01-25T06:31:22.774 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2023-01-25T06:31:22.775 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2023-01-25T06:31:22.830 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2023-01-25T06:31:22.833 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2023-01-25T06:31:22.961 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2023-01-25T06:31:23.121 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:23.426 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:23.924 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 1/1 2023-01-25T06:31:23.925 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:23.925 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2023-01-25T06:31:23.925 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:18.0.0-1963.g61996d73.el8.x86_64 2023-01-25T06:31:23.925 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:23.926 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2023-01-25T06:31:24.101 DEBUG:teuthology.parallel:result is None 2023-01-25T06:31:24.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo lsb_release -is 2023-01-25T06:31:24.141 INFO:teuthology.orchestra.run.smithi062.stdout:CentOSStream 2023-01-25T06:31:24.142 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:31:24.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=61996d73ab738798e4ee0055c86457f5c459da4a 2023-01-25T06:31:24.290 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T06:31:24.313 INFO:teuthology.orchestra.run.smithi062.stdout:18.0.0-1963.g61996d73.el8 2023-01-25T06:31:24.313 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1963.g61996d73.el8 2023-01-25T06:31:24.314 INFO:teuthology.task.install:The correct ceph version 18.0.0-1963.g61996d73 is installed. 2023-01-25T06:31:24.314 DEBUG:teuthology.orchestra.run.smithi110:> sudo lsb_release -is 2023-01-25T06:31:24.356 INFO:teuthology.orchestra.run.smithi110.stdout:CentOSStream 2023-01-25T06:31:24.358 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T06:31:24.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=61996d73ab738798e4ee0055c86457f5c459da4a 2023-01-25T06:31:24.506 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T06:31:24.530 INFO:teuthology.orchestra.run.smithi110.stdout:18.0.0-1963.g61996d73.el8 2023-01-25T06:31:24.530 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1963.g61996d73.el8 2023-01-25T06:31:24.531 INFO:teuthology.task.install:The correct ceph version 18.0.0-1963.g61996d73 is installed. 2023-01-25T06:31:24.531 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T06:31:24.531 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:24.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T06:31:24.560 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:24.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T06:31:24.598 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T06:31:24.598 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:24.599 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T06:31:24.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T06:31:24.695 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:24.696 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T06:31:24.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T06:31:24.791 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T06:31:24.791 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:24.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T06:31:24.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T06:31:24.887 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:24.887 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T06:31:24.914 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T06:31:24.980 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T06:31:25.041 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T06:31:25.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2023-01-25T06:31:25.044 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /var/log/ceph 2023-01-25T06:31:25.071 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T06:31:25.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T06:31:25.113 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T06:31:25.140 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T06:31:25.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T06:31:25.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T06:31:25.219 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T06:31:25.220 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2023-01-25T06:31:25.220 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi/7136917', 'branch': 'wip-yuri2-testing-2023-01-23-0928', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136917', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:22:09-rados-wip-yuri2-testing-2023-01-23-0928-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}, {'volumes': {'size': 50}}, {'machine': {'ram': 15000}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2023-01-23-0928'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri2-testing-2023-01-23-0928', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri2-testing-2023-01-23-0928', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_61996d73ab738798e4ee0055c86457f5c459da4a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '61996d73ab738798e4ee0055c86457f5c459da4a', 'targets': {'smithi062.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiU0p+LdQaepzlZFUYWJhRyMPNBzRtv7db6XpsJfTBfiPKc8GdAc7yW/vWWSfmFDMv7DoNgFnm+O5RULi5W4JL74+P5URruPK/lcqnKJvgvQnTnKIrW2pu+fLmiDGDd1Etabjpee/15u9/8UeGdASj0c2UIqV1+6PW0DcFMBOoq+Izlv4VYgRYHuDzvkqjiOKxDcB66yb9PBB24UyJUV2dDjApT/c0lJoNTeFxANb8yuRoSkXU34rBSGaW3oFtims/HdOoSjDojvmBnMMEMDgrPJIXtIowiaNPmKl3Rd5QbxMdW2TGxtry42DQK1AGHnmueJ6Wy9HJWTbgqiDlzRjWYeTDSL1PlOjOZR7+olVLOajyWNvhc7tgjovdgAZGjbFkjCG5E13v6K8vNTD6t2qgadKFtSzgwl8REhdMmKxw/jLzC1Bq4bVJ3CYTak4g47qpfLZvmTdi30K3aYQFEzLokgcJXmxZZi0tQg90EE+RNMfMyFgRxpacHHOgmxpvMZE=', 'smithi110.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGLYopS5UbZEcsh1MaAdR9N9s0cRRdPmnLGpOGgbaFTBk49irNW34NokdIwBn4orfrTkahRgBoKRuY8Bbi6rc2H+SCKEN0QfIZyY3PypkI+UicQTSCSMHigiXXgFvuNKEpifNMIdTvOTPkxFVaBQdL76tB5Hr59v187h98ZOdhbgdaqRHF9WWGQ1sHA3G9fHwiVDmPmKK2OljIkk1n/nI/L6FCr03FykCXh5ImotKfng5r7obxssIezrWNm6GZgVVpoD4YWrp+FQR5LLNCb8kJv+16uolb9Jm6LD1eA4ufu49A4xmoCTY936oqvkpOJbiH7xRHyQ23KXl9ba8P+FQVrd+q4rttf1U3Zzmztv9I72R/EsVpScPlG1zLZWsha7E5hHiRDiru5DoHVOK/Wb/t4xMzw0TNXaA1V74JOvdovxbhZ87nfqmMjeBTFFBKwiGZEJFqBdTiSYtUuIdhpIJv4bd2QEDkOx0LbCMPDuqzMLdx80eOr5PoPXlhwtL1Mh8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 0', 'sudo ceph osd pool set cache min_write_recency_for_promote 0']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pool_snaps': True, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:22:09', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T06:31:25.221 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T06:31:25.250 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T06:31:25.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T06:31:25.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T06:31:25.336 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:25.336 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T06:31:25.392 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T06:31:25.393 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2023-01-25T06:31:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T06:31:25.450 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:25.450 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 370152 Links: 1 2023-01-25T06:31:25.451 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:25.451 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:25.451 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-25 06:31:19.396397292 +0000 2023-01-25T06:31:25.451 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-25 06:27:32.336957817 +0000 2023-01-25T06:31:25.451 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-25 06:27:32.336957817 +0000 2023-01-25T06:31:25.452 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-25T06:31:25.452 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T06:31:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-25T06:31:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-25T06:31:25.517 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000231436 s, 2.2 MB/s 2023-01-25T06:31:25.518 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T06:31:25.577 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2023-01-25T06:31:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T06:31:25.635 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:25.636 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 372254 Links: 1 2023-01-25T06:31:25.636 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:25.636 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:25.637 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-25 06:31:19.396397292 +0000 2023-01-25T06:31:25.637 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-25 06:27:32.854949804 +0000 2023-01-25T06:31:25.637 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-25 06:27:32.854949804 +0000 2023-01-25T06:31:25.637 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-25T06:31:25.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T06:31:25.701 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-25T06:31:25.702 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-25T06:31:25.702 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000206357 s, 2.5 MB/s 2023-01-25T06:31:25.703 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T06:31:25.762 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2023-01-25T06:31:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T06:31:25.820 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 373133 Links: 1 2023-01-25T06:31:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-25 06:31:19.396397292 +0000 2023-01-25T06:31:25.821 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-25 06:27:33.370941821 +0000 2023-01-25T06:31:25.822 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-25 06:27:33.370941821 +0000 2023-01-25T06:31:25.822 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-25T06:31:25.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T06:31:25.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-25T06:31:25.887 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-25T06:31:25.888 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000247329 s, 2.1 MB/s 2023-01-25T06:31:25.889 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T06:31:25.948 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2023-01-25T06:31:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T06:31:26.007 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:26.007 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 6h/6d Inode: 371577 Links: 1 2023-01-25T06:31:26.008 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:26.008 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:26.008 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2023-01-25 06:31:19.396397292 +0000 2023-01-25T06:31:26.008 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2023-01-25 06:27:33.891933762 +0000 2023-01-25T06:31:26.008 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2023-01-25 06:27:33.891933762 +0000 2023-01-25T06:31:26.009 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: - 2023-01-25T06:31:26.009 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T06:31:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2023-01-25T06:31:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2023-01-25T06:31:26.074 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.00017654 s, 2.9 MB/s 2023-01-25T06:31:26.075 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T06:31:26.134 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:26.135 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:26.135 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T06:31:26.154 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T06:31:26.155 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2023-01-25T06:31:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T06:31:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:26.214 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 367270 Links: 1 2023-01-25T06:31:26.214 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:26.214 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:26.215 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-01-25 06:31:03.280911492 +0000 2023-01-25T06:31:26.215 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-01-25 06:27:24.482914136 +0000 2023-01-25T06:31:26.215 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-01-25 06:27:24.482914136 +0000 2023-01-25T06:31:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-01-25T06:31:26.216 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T06:31:26.280 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-01-25T06:31:26.281 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-01-25T06:31:26.281 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000108195 s, 4.7 MB/s 2023-01-25T06:31:26.282 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T06:31:26.341 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2023-01-25T06:31:26.400 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T06:31:26.401 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:26.401 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 367453 Links: 1 2023-01-25T06:31:26.402 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:26.402 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:26.402 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-01-25 06:31:03.280911492 +0000 2023-01-25T06:31:26.403 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-01-25 06:27:24.978906278 +0000 2023-01-25T06:31:26.403 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-01-25 06:27:24.978906278 +0000 2023-01-25T06:31:26.404 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-01-25T06:31:26.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T06:31:26.466 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-01-25T06:31:26.466 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-01-25T06:31:26.467 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.5438e-05 s, 6.0 MB/s 2023-01-25T06:31:26.468 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T06:31:26.526 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2023-01-25T06:31:26.585 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T06:31:26.585 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:26.585 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 375111 Links: 1 2023-01-25T06:31:26.586 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:26.586 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:26.586 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-01-25 06:31:03.281911476 +0000 2023-01-25T06:31:26.586 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-01-25 06:27:25.468898516 +0000 2023-01-25T06:31:26.587 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-01-25 06:27:25.468898516 +0000 2023-01-25T06:31:26.587 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-01-25T06:31:26.587 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T06:31:26.650 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-01-25T06:31:26.650 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-01-25T06:31:26.650 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000121886 s, 4.2 MB/s 2023-01-25T06:31:26.651 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T06:31:26.711 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2023-01-25T06:31:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T06:31:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T06:31:26.769 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 375261 Links: 1 2023-01-25T06:31:26.769 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T06:31:26.770 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T06:31:26.770 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-01-25 06:31:03.281911476 +0000 2023-01-25T06:31:26.770 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-01-25 06:27:25.964890659 +0000 2023-01-25T06:31:26.770 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-01-25 06:27:25.964890659 +0000 2023-01-25T06:31:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-01-25T06:31:26.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T06:31:26.834 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-01-25T06:31:26.834 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-01-25T06:31:26.835 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000115664 s, 4.4 MB/s 2023-01-25T06:31:26.836 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T06:31:26.894 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:26.894 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T06:31:26.894 INFO:tasks.ceph:Generating config... 2023-01-25T06:31:26.896 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-25T06:31:26.896 INFO:tasks.ceph:[osd] bdev async discard = True 2023-01-25T06:31:26.896 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-01-25T06:31:26.897 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-01-25T06:31:26.897 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-25T06:31:26.897 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-25T06:31:26.898 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T06:31:26.898 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-25T06:31:26.898 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-25T06:31:26.899 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T06:31:26.899 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T06:31:26.899 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-25T06:31:26.899 INFO:tasks.ceph:[osd] filestore queue throttle high multiple = 2 2023-01-25T06:31:26.900 INFO:tasks.ceph:[osd] filestore queue throttle max multiple = 10 2023-01-25T06:31:26.900 INFO:tasks.ceph:[osd] journal throttle high multiple = 2 2023-01-25T06:31:26.900 INFO:tasks.ceph:[osd] journal throttle max multiple = 10 2023-01-25T06:31:26.901 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-25T06:31:26.901 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-25T06:31:26.901 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-25T06:31:26.901 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-01-25T06:31:26.901 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T06:31:26.902 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T06:31:26.902 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-25T06:31:26.902 INFO:tasks.ceph:[osd] osd max backfills = 9 2023-01-25T06:31:26.902 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-25T06:31:26.903 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T06:31:26.903 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-25T06:31:26.903 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T06:31:26.903 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T06:31:26.904 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2023-01-25T06:31:26.904 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2023-01-25T06:31:26.904 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-25T06:31:26.904 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-25T06:31:26.905 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-25T06:31:26.905 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T06:31:26.905 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-25T06:31:26.906 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-01-25T06:31:26.906 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2023-01-25T06:31:26.906 INFO:tasks.ceph:[global] ms tcp read timeout = 5 2023-01-25T06:31:26.907 INFO:tasks.ceph:[global] ms type = async 2023-01-25T06:31:26.907 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2023-01-25T06:31:26.907 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2023-01-25T06:31:26.908 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2023-01-25T06:31:26.908 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2023-01-25T06:31:26.908 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-25T06:31:26.908 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2023-01-25T06:31:26.909 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T06:31:26.909 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T06:31:26.909 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T06:31:26.909 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T06:31:26.910 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T06:31:26.910 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T06:31:26.910 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T06:31:26.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T06:31:26.970 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T06:31:26.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T06:31:27.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T06:31:27.048 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.62:6789]'), ('mon.c', '[v1:172.21.15.62:6790]'), ('mon.b', '[v1:172.21.15.110:6789]')] 2023-01-25T06:31:27.048 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.62:6789],[v1:172.21.15.62:6790],[v1:172.21.15.110:6789]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-25T06:31:27.049 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:27.050 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T06:31:27.106 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.62:6789]' --addv c '[v1:172.21.15.62:6790]' --addv b '[v1:172.21.15.110:6789]' --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:27.196 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:27.196 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: generated fsid b10aa3db-f868-4fb2-8582-d4662bf923a5 2023-01-25T06:31:27.196 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = pacific 2023-01-25T06:31:27.197 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 0 2023-01-25T06:31:27.197 INFO:teuthology.orchestra.run.smithi062.stdout:fsid b10aa3db-f868-4fb2-8582-d4662bf923a5 2023-01-25T06:31:27.197 INFO:teuthology.orchestra.run.smithi062.stdout:last_changed 2023-01-25T06:31:27.196131+0000 2023-01-25T06:31:27.197 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-01-25T06:31:27.196131+0000 2023-01-25T06:31:27.197 INFO:teuthology.orchestra.run.smithi062.stdout:min_mon_release 16 (pacific) 2023-01-25T06:31:27.198 INFO:teuthology.orchestra.run.smithi062.stdout:election_strategy: 3 2023-01-25T06:31:27.198 INFO:teuthology.orchestra.run.smithi062.stdout:0: v1:172.21.15.62:6789/0 mon.a 2023-01-25T06:31:27.198 INFO:teuthology.orchestra.run.smithi062.stdout:1: v1:172.21.15.110:6789/0 mon.b 2023-01-25T06:31:27.198 INFO:teuthology.orchestra.run.smithi062.stdout:2: v1:172.21.15.62:6790/0 mon.c 2023-01-25T06:31:27.198 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-25T06:31:27.199 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T06:31:27.218 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b10aa3db-f868-4fb2-8582-d4662bf923a5... 2023-01-25T06:31:27.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T06:31:27.260 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T06:31:27.304 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2023-01-25T06:31:27.304 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2023-01-25T06:31:27.304 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T06:31:27.304 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.305 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2023-01-25T06:31:27.306 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2023-01-25T06:31:27.306 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.306 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2023-01-25T06:31:27.306 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2023-01-25T06:31:27.306 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.307 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T06:31:27.307 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2023-01-25T06:31:27.307 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.307 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T06:31:27.307 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T06:31:27.308 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.308 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2023-01-25T06:31:27.308 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.308 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2023-01-25T06:31:27.308 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2023-01-25T06:31:27.309 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T06:31:27.310 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T06:31:27.310 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2023-01-25T06:31:27.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.310 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T06:31:27.310 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2023-01-25T06:31:27.311 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2023-01-25T06:31:27.312 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2023-01-25T06:31:27.312 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "[v1:172.21.15.62:6789],[v1:172.21.15.62:6790],[v1:172.21.15.110:6789]" 2023-01-25T06:31:27.312 INFO:teuthology.orchestra.run.smithi062.stdout: mon client directed command retry = 5 2023-01-25T06:31:27.312 INFO:teuthology.orchestra.run.smithi062.stdout: mon election default strategy = 3 2023-01-25T06:31:27.312 INFO:teuthology.orchestra.run.smithi062.stdout: ms bind msgr2 = False 2023-01-25T06:31:27.313 INFO:teuthology.orchestra.run.smithi062.stdout: ms inject socket failures = 2500 2023-01-25T06:31:27.313 INFO:teuthology.orchestra.run.smithi062.stdout: ms tcp read timeout = 5 2023-01-25T06:31:27.313 INFO:teuthology.orchestra.run.smithi062.stdout: ms type = async 2023-01-25T06:31:27.313 INFO:teuthology.orchestra.run.smithi062.stdout: osd_async_recovery_min_cost = 1 2023-01-25T06:31:27.313 INFO:teuthology.orchestra.run.smithi062.stdout: osd_max_pg_log_entries = 2 2023-01-25T06:31:27.314 INFO:teuthology.orchestra.run.smithi062.stdout: osd_min_pg_log_entries = 1 2023-01-25T06:31:27.314 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pg_log_trim_min = 0 2023-01-25T06:31:27.314 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_min_size = 2 2023-01-25T06:31:27.314 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_size = 3 2023-01-25T06:31:27.314 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = b10aa3db-f868-4fb2-8582-d4662bf923a5 2023-01-25T06:31:27.315 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.315 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2023-01-25T06:31:27.315 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2023-01-25T06:31:27.315 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.316 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2023-01-25T06:31:27.316 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 120 2023-01-25T06:31:27.316 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.316 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2023-01-25T06:31:27.316 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2023-01-25T06:31:27.317 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.317 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2023-01-25T06:31:27.317 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2023-01-25T06:31:27.317 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2023-01-25T06:31:27.317 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.318 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2023-01-25T06:31:27.318 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2023-01-25T06:31:27.318 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.318 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2023-01-25T06:31:27.318 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2023-01-25T06:31:27.319 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2023-01-25T06:31:27.320 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.320 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2023-01-25T06:31:27.320 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2023-01-25T06:31:27.320 INFO:teuthology.orchestra.run.smithi062.stdout: debug monc = 20 2023-01-25T06:31:27.320 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2023-01-25T06:31:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2023-01-25T06:31:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2023-01-25T06:31:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2023-01-25T06:31:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2023-01-25T06:31:27.321 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore zero block detection = True 2023-01-25T06:31:27.322 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 20 2023-01-25T06:31:27.322 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 20 2023-01-25T06:31:27.322 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-25T06:31:27.322 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2023-01-25T06:31:27.322 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 10 2023-01-25T06:31:27.323 INFO:teuthology.orchestra.run.smithi062.stdout: filestore queue throttle high multiple = 2 2023-01-25T06:31:27.323 INFO:teuthology.orchestra.run.smithi062.stdout: filestore queue throttle max multiple = 10 2023-01-25T06:31:27.323 INFO:teuthology.orchestra.run.smithi062.stdout: journal throttle high multiple = 2 2023-01-25T06:31:27.323 INFO:teuthology.orchestra.run.smithi062.stdout: journal throttle max multiple = 10 2023-01-25T06:31:27.323 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T06:31:27.324 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2023-01-25T06:31:27.324 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2023-01-25T06:31:27.324 INFO:teuthology.orchestra.run.smithi062.stdout: osd blocked scrub grace period = 3600 2023-01-25T06:31:27.324 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify cached snaps = True 2023-01-25T06:31:27.324 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify missing on start = True 2023-01-25T06:31:27.325 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2023-01-25T06:31:27.325 INFO:teuthology.orchestra.run.smithi062.stdout: osd max backfills = 9 2023-01-25T06:31:27.325 INFO:teuthology.orchestra.run.smithi062.stdout: osd max markdown count = 1000 2023-01-25T06:31:27.325 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock override recovery settings = True 2023-01-25T06:31:27.325 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2023-01-25T06:31:27.326 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue = debug_random 2023-01-25T06:31:27.326 INFO:teuthology.orchestra.run.smithi062.stdout: osd op queue cut off = debug_random 2023-01-25T06:31:27.326 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub min interval = 60 2023-01-25T06:31:27.326 INFO:teuthology.orchestra.run.smithi062.stdout: osd shutdown pgref assert = True 2023-01-25T06:31:27.326 INFO:teuthology.orchestra.run.smithi062.stdout: osd_max_pg_log_entries = 3000 2023-01-25T06:31:27.327 INFO:teuthology.orchestra.run.smithi062.stdout: osd_min_pg_log_entries = 3000 2023-01-25T06:31:27.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.327 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2023-01-25T06:31:27.327 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-25T06:31:27.327 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2023-01-25T06:31:27.328 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-01-25T06:31:27.328 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-01-25T06:31:27.328 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:31:27.328 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:31:27.328 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2023-01-25T06:31:27.329 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.329 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2023-01-25T06:31:27.329 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2023-01-25T06:31:27.329 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2023-01-25T06:31:27.329 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2023-01-25T06:31:27.330 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2023-01-25T06:31:27.331 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:31:27.331 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.331 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T06:31:27.331 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T06:31:27.331 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T06:31:27.332 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.332 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T06:31:27.332 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T06:31:27.332 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T06:31:27.332 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: mon scrub interval = 300 2023-01-25T06:31:27.333 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:31:27.334 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2023-01-25T06:31:27.334 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2023-01-25T06:31:27.334 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2023-01-25T06:31:27.334 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2023-01-25T06:31:27.334 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T06:31:27.335 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T06:31:27.335 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2023-01-25T06:31:27.335 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2023-01-25T06:31:27.335 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2023-01-25T06:31:27.337 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2023-01-25T06:31:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: chdir = "" 2023-01-25T06:31:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T06:31:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: auth supported = cephx 2023-01-25T06:31:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: filestore xattr use omap = true 2023-01-25T06:31:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: mon clock drift allowed = 1.000 2023-01-25T06:31:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: osd crush chooseleaf type = 0 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: auth debug = true 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on old message = true 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on bug = true 2023-01-25T06:31:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T06:31:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: mon pg warn max object skew = 0 2023-01-25T06:31:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T06:31:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T06:31:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default size = 2 2023-01-25T06:31:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow primary affinity = true 2023-01-25T06:31:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow pg remap = true 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on legacy crush tunables = false 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on crush straw calc version zero = false 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on no sortbitwise = false 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on osd down out interval zero = false 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on too few osds = false 2023-01-25T06:31:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T06:31:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T06:31:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: mon_allow_pool_size_one = true 2023-01-25T06:31:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T06:31:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: osd default data pool replay window = 5 2023-01-25T06:31:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: mon allow pool delete = true 2023-01-25T06:31:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: mon cluster log file level = debug 2023-01-25T06:31:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: debug asserts on shutdown = true 2023-01-25T06:31:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: mon health detail to clog = false 2023-01-25T06:31:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: mon host = "[v1:172.21.15.62:6789],[v1:172.21.15.62:6790],[v1:172.21.15.110:6789]" 2023-01-25T06:31:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: mon client directed command retry = 5 2023-01-25T06:31:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: mon election default strategy = 3 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: ms bind msgr2 = False 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: ms inject socket failures = 2500 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: ms tcp read timeout = 5 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: ms type = async 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: osd_async_recovery_min_cost = 1 2023-01-25T06:31:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: osd_max_pg_log_entries = 2 2023-01-25T06:31:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: osd_min_pg_log_entries = 1 2023-01-25T06:31:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pg_log_trim_min = 0 2023-01-25T06:31:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_min_size = 2 2023-01-25T06:31:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_size = 3 2023-01-25T06:31:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = b10aa3db-f868-4fb2-8582-d4662bf923a5 2023-01-25T06:31:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.348 INFO:teuthology.orchestra.run.smithi110.stdout:[osd] 2023-01-25T06:31:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: osd journal size = 100 2023-01-25T06:31:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub load threshold = 5.0 2023-01-25T06:31:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub max interval = 120 2023-01-25T06:31:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: osd recover clone overlap = true 2023-01-25T06:31:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: osd recovery max chunk = 1048576 2023-01-25T06:31:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug shutdown = true 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug op order = true 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify stray on activate = true 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: osd open classes on start = true 2023-01-25T06:31:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug pg log writeout = true 2023-01-25T06:31:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: osd deep scrub update digest min age = 30 2023-01-25T06:31:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: osd map max advance = 10 2023-01-25T06:31:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: journal zero on create = true 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: filestore ondisk finisher threads = 3 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: filestore apply finisher threads = 3 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: bdev debug aio = true 2023-01-25T06:31:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug misdirected ops = true 2023-01-25T06:31:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: debug monc = 20 2023-01-25T06:31:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: bdev async discard = True 2023-01-25T06:31:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: bdev enable discard = True 2023-01-25T06:31:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore allocator = bitmap 2023-01-25T06:31:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore block size = 96636764160 2023-01-25T06:31:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore fsck on mount = True 2023-01-25T06:31:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore zero block detection = True 2023-01-25T06:31:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluefs = 20 2023-01-25T06:31:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluestore = 20 2023-01-25T06:31:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2023-01-25T06:31:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: debug osd = 20 2023-01-25T06:31:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: debug rocksdb = 10 2023-01-25T06:31:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: filestore queue throttle high multiple = 2 2023-01-25T06:31:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: filestore queue throttle max multiple = 10 2023-01-25T06:31:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: journal throttle high multiple = 2 2023-01-25T06:31:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: journal throttle max multiple = 10 2023-01-25T06:31:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd backfillfull_ratio = 0.85 2023-01-25T06:31:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd full ratio = 0.9 2023-01-25T06:31:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd nearfull ratio = 0.8 2023-01-25T06:31:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: osd blocked scrub grace period = 3600 2023-01-25T06:31:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify cached snaps = True 2023-01-25T06:31:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify missing on start = True 2023-01-25T06:31:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: osd failsafe full ratio = 0.95 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd max backfills = 9 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd max markdown count = 1000 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock override recovery settings = True 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd objectstore = bluestore 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd op queue = debug_random 2023-01-25T06:31:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: osd op queue cut off = debug_random 2023-01-25T06:31:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub min interval = 60 2023-01-25T06:31:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: osd shutdown pgref assert = True 2023-01-25T06:31:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: osd_max_pg_log_entries = 3000 2023-01-25T06:31:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: osd_min_pg_log_entries = 3000 2023-01-25T06:31:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.360 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr] 2023-01-25T06:31:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2023-01-25T06:31:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: debug mgr = 20 2023-01-25T06:31:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2023-01-25T06:31:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: mgr/telemetry/nag = false 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout:[mon] 2023-01-25T06:31:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2023-01-25T06:31:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2023-01-25T06:31:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: debug paxos = 20 2023-01-25T06:31:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2023-01-25T06:31:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: mon data avail warn = 5 2023-01-25T06:31:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: mon mgr mkfs grace = 240 2023-01-25T06:31:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2023-01-25T06:31:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd reporter subtree level = osd 2023-01-25T06:31:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd prime pg temp = true 2023-01-25T06:31:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2023-01-25T06:31:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T06:31:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T06:31:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T06:31:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: # 1m isn't quite enough 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: mon_down_mkfs_grace = 2m 2023-01-25T06:31:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_filestore_osds = false 2023-01-25T06:31:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: mon scrub interval = 300 2023-01-25T06:31:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-01-25T06:31:27.366 INFO:teuthology.orchestra.run.smithi110.stdout:[client] 2023-01-25T06:31:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: rgw cache enabled = true 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable ops log = true 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable usage log = true 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.a] 2023-01-25T06:31:27.367 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.c] 2023-01-25T06:31:27.368 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.b] 2023-01-25T06:31:27.368 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T06:31:27.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T06:31:27.427 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T06:31:27.427 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:27.427 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T06:31:27.446 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:27.446 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T06:31:27.504 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:31:27.504 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:27.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T06:31:27.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T06:31:27.581 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:27.582 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:27.639 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:31:27.640 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:27.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T06:31:27.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T06:31:27.678 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:27.678 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:27.736 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T06:31:27.736 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T06:31:27.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T06:31:27.792 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-25T06:31:27.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T06:31:27.850 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T06:31:27.853 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T06:31:27.853 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T06:31:27.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T06:31:27.899 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T06:31:27.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-25T06:31:27.955 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-25T06:31:27.967 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T06:31:27.968 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-25T06:31:27.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T06:31:27.996 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:27.996 INFO:tasks.ceph:role: osd.0 2023-01-25T06:31:27.997 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:31:27.997 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T06:31:28.071 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:28.071 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:28.071 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:28.072 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:28.072 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:28.072 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:28.072 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:28.072 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:28.073 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:28.073 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:28.495 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-25T06:31:28.497 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-25T06:31:28.497 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T06:31:28.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T06:31:28.671 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T06:31:28.739 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:28.740 INFO:tasks.ceph:role: osd.1 2023-01-25T06:31:28.740 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:31:28.741 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T06:31:28.814 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:28.814 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:28.815 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:28.815 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:28.815 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:28.815 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:28.815 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:28.816 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:28.816 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:28.816 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:29.244 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-25T06:31:29.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-25T06:31:29.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T06:31:29.335 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T06:31:29.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T06:31:29.437 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:29.437 INFO:tasks.ceph:role: osd.2 2023-01-25T06:31:29.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:31:29.438 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T06:31:29.512 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:29.513 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:29.513 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:29.513 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:29.513 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:29.514 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:29.514 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:29.514 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:29.514 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:29.514 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:29.954 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-25T06:31:29.956 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-25T06:31:29.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T06:31:30.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T06:31:30.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T06:31:30.174 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:30.175 INFO:tasks.ceph:role: osd.3 2023-01-25T06:31:30.175 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2023-01-25T06:31:30.176 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T06:31:30.251 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:30.251 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:30.252 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:30.252 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:30.252 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:30.252 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:30.252 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:30.253 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:30.253 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:30.253 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:30.687 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2023-01-25T06:31:30.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2023-01-25T06:31:30.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T06:31:30.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T06:31:30.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:31.005 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.005+0000 7f367e06e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T06:31:31.008 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.008+0000 7f367e06e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T06:31:31.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.008+0000 7f367e06e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:31:31.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.008+0000 7f367e06e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:31:31.009 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.008+0000 7f367e06e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T06:31:31.012 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.012+0000 7f367e06e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T06:31:31.536 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:31.535+0000 7f367e06e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:32.296 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:32.296+0000 7f367e06e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:33.063 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.062+0000 7f367e06e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:33.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T06:31:33.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:33.992 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.991+0000 7fc4fb9075c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T06:31:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.991+0000 7fc4fb9075c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T06:31:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.991+0000 7fc4fb9075c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:31:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.991+0000 7fc4fb9075c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:31:33.993 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.991+0000 7fc4fb9075c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T06:31:33.994 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:33.992+0000 7fc4fb9075c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T06:31:34.516 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:34.515+0000 7fc4fb9075c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:35.279 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:35.279+0000 7fc4fb9075c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:36.042 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.041+0000 7fc4fb9075c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:36.833 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T06:31:36.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:36.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T06:31:36.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T06:31:36.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:31:36.959 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:31:36.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T06:31:36.960 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:36.958+0000 7fd773a6c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T06:31:37.492 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:37.491+0000 7fd773a6c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:38.253 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:38.252+0000 7fd773a6c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:39.015 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.014+0000 7fd773a6c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:39.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T06:31:39.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:39.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.937+0000 7f59891a05c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T06:31:39.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.937+0000 7f59891a05c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T06:31:39.938 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.937+0000 7f59891a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T06:31:39.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.937+0000 7f59891a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T06:31:39.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.937+0000 7f59891a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T06:31:39.939 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:39.938+0000 7f59891a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T06:31:40.462 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:40.462+0000 7f59891a05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:41.224 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:41.223+0000 7f59891a05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:41.986 INFO:teuthology.orchestra.run.smithi062.stderr:2023-01-25T06:31:41.985+0000 7f59891a05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:42.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T06:31:42.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T06:31:42.846 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:42.847 INFO:tasks.ceph:role: osd.4 2023-01-25T06:31:42.847 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:31:42.847 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T06:31:42.915 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:42.916 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:42.916 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:42.916 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:42.916 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:42.917 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:42.917 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:42.917 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:42.938 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2023-01-25T06:31:42.939 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2023-01-25T06:31:42.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-25T06:31:43.021 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T06:31:43.094 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T06:31:43.161 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:43.162 INFO:tasks.ceph:role: osd.5 2023-01-25T06:31:43.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:31:43.162 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T06:31:43.229 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:43.230 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:43.230 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:43.230 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:43.230 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:43.230 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:43.231 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:43.231 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:43.231 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:43.231 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:43.252 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2023-01-25T06:31:43.254 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2023-01-25T06:31:43.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-25T06:31:43.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T06:31:43.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T06:31:43.472 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:43.473 INFO:tasks.ceph:role: osd.6 2023-01-25T06:31:43.473 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:31:43.474 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T06:31:43.540 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:43.540 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:43.540 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:43.541 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:43.541 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:43.541 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:43.541 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:43.541 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:43.542 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:43.544 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:43.563 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2023-01-25T06:31:43.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2023-01-25T06:31:43.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-25T06:31:43.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-25T06:31:43.716 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T06:31:43.783 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-25T06:31:43.784 INFO:tasks.ceph:role: osd.7 2023-01-25T06:31:43.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com 2023-01-25T06:31:43.784 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T06:31:43.850 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T06:31:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T06:31:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T06:31:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T06:31:43.851 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T06:31:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2023-01-25T06:31:43.851 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T06:31:43.851 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T06:31:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T06:31:43.852 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T06:31:43.873 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2023-01-25T06:31:43.874 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2023-01-25T06:31:43.875 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-25T06:31:43.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-25T06:31:44.025 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.132+0000 7f7fa60e95c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T06:31:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.132+0000 7f7fa60e95c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T06:31:44.133 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.132+0000 7f7fa60e95c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T06:31:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.132+0000 7f7fa60e95c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T06:31:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.132+0000 7f7fa60e95c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T06:31:44.134 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.133+0000 7f7fa60e95c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T06:31:44.659 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:44.659+0000 7f7fa60e95c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:45.424 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:45.424+0000 7f7fa60e95c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:46.190 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:46.190+0000 7f7fa60e95c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:46.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T06:31:47.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:47.099 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.098+0000 7fc5c84515c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T06:31:47.099 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.098+0000 7fc5c84515c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T06:31:47.099 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.098+0000 7fc5c84515c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T06:31:47.099 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.098+0000 7fc5c84515c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T06:31:47.099 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.098+0000 7fc5c84515c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T06:31:47.100 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.099+0000 7fc5c84515c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T06:31:47.622 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:47.622+0000 7fc5c84515c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:48.385 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:48.385+0000 7fc5c84515c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:49.148 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:49.148+0000 7fc5c84515c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:49.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T06:31:49.971 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:50.062 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T06:31:50.063 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T06:31:50.063 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T06:31:50.063 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T06:31:50.063 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T06:31:50.063 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.060+0000 7f2ca53d15c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T06:31:50.582 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:50.582+0000 7f2ca53d15c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:51.344 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:51.344+0000 7f2ca53d15c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:52.112+0000 7f2ca53d15c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:52.903 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T06:31:52.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.021+0000 7f614b6135c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T06:31:53.022 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.021+0000 7f614b6135c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T06:31:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.021+0000 7f614b6135c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T06:31:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.021+0000 7f614b6135c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T06:31:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.021+0000 7f614b6135c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T06:31:53.023 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.022+0000 7f614b6135c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T06:31:53.544 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:53.543+0000 7f614b6135c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:54.306 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:54.305+0000 7f614b6135c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:55.068 INFO:teuthology.orchestra.run.smithi110.stderr:2023-01-25T06:31:55.067+0000 7f614b6135c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T06:31:55.856 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T06:31:55.885 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T06:31:55.885 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:55.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-25T06:31:55.919 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:55.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T06:31:55.991 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:55.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T06:31:56.062 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:56.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T06:31:56.132 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:56.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T06:31:56.201 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T06:31:56.231 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T06:31:56.299 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.299 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T06:31:56.368 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T06:31:56.436 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.437 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T06:31:56.504 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:31:56.505 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T06:31:56.525 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:31:56.525 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-25T06:31:56.563 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T06:31:56.563 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T06:31:56.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T06:31:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2023-01-25T06:31:56.606 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC/zNBjsmM7LxAA7PTF1a3+M6iuYceyRdFv/A== 2023-01-25T06:31:56.606 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2023-01-25T06:31:56.607 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDDzNBjxjJVABAACeQIuZx0gPaldKJRIEaFNw== 2023-01-25T06:31:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2023-01-25T06:31:56.607 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDFzNBj3lUeOxAA0pFAJAY3x9L+jQV9FoO+wA== 2023-01-25T06:31:56.607 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2023-01-25T06:31:56.608 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDIzNBjiB0mORAAnAr+2bil5+IFVxhq2+OZZg== 2023-01-25T06:31:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2023-01-25T06:31:56.608 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDLzNBjtXfpNxAA2XNBO8o+7pkXAYR/964FKw== 2023-01-25T06:31:56.608 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2023-01-25T06:31:56.609 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC/zNBj6Aa7MhAANrs2iAhAUF2s8aEUxxXLtg== 2023-01-25T06:31:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2023-01-25T06:31:56.609 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDQzNBj3nLtBxAAGAL61eEjciNe8pqJtxg6sg== 2023-01-25T06:31:56.609 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2023-01-25T06:31:56.609 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDTzNBjbtXkBRAAEAS+rCbkEBQ+bhHVsI9Qdg== 2023-01-25T06:31:56.610 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2023-01-25T06:31:56.610 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDWzNBjCf6eAxAADPnIcIUpjLd9B8KhMfgwJQ== 2023-01-25T06:31:56.610 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2023-01-25T06:31:56.610 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQDZzNBjxjpQARAAp00SixDf/6JmRLaIYFml1A== 2023-01-25T06:31:56.610 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2023-01-25T06:31:56.611 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC/zNBjB7KiNRAAx8U07gQm+gaq4p65MZXdwA== 2023-01-25T06:31:56.611 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2023-01-25T06:31:56.611 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQC/zNBjI1f9OBAA/xa7gbcB70lUmAGqXH76sA== 2023-01-25T06:31:56.628 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.y] 2023-01-25T06:31:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQC/zNBjsmM7LxAA7PTF1a3+M6iuYceyRdFv/A== 2023-01-25T06:31:56.629 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.0] 2023-01-25T06:31:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDDzNBjxjJVABAACeQIuZx0gPaldKJRIEaFNw== 2023-01-25T06:31:56.629 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.1] 2023-01-25T06:31:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDFzNBj3lUeOxAA0pFAJAY3x9L+jQV9FoO+wA== 2023-01-25T06:31:56.630 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.2] 2023-01-25T06:31:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDIzNBjiB0mORAAnAr+2bil5+IFVxhq2+OZZg== 2023-01-25T06:31:56.630 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.3] 2023-01-25T06:31:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDLzNBjtXfpNxAA2XNBO8o+7pkXAYR/964FKw== 2023-01-25T06:31:56.630 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.x] 2023-01-25T06:31:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQC/zNBj6Aa7MhAANrs2iAhAUF2s8aEUxxXLtg== 2023-01-25T06:31:56.631 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.4] 2023-01-25T06:31:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDQzNBj3nLtBxAAGAL61eEjciNe8pqJtxg6sg== 2023-01-25T06:31:56.631 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.5] 2023-01-25T06:31:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDTzNBjbtXkBRAAEAS+rCbkEBQ+bhHVsI9Qdg== 2023-01-25T06:31:56.632 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.6] 2023-01-25T06:31:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDWzNBjCf6eAxAADPnIcIUpjLd9B8KhMfgwJQ== 2023-01-25T06:31:56.632 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.7] 2023-01-25T06:31:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQDZzNBjxjpQARAAp00SixDf/6JmRLaIYFml1A== 2023-01-25T06:31:56.632 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2023-01-25T06:31:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQC/zNBjB7KiNRAAx8U07gQm+gaq4p65MZXdwA== 2023-01-25T06:31:56.633 INFO:teuthology.orchestra.run.smithi110.stdout:[client.1] 2023-01-25T06:31:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQC/zNBjI1f9OBAA/xa7gbcB70lUmAGqXH76sA== 2023-01-25T06:31:56.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T06:31:56.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T06:31:56.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.730 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.786 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.841 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:56.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T06:31:56.952 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T06:31:57.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.008 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.061 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.063 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.119 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.175 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T06:31:57.229 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:31:57.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:31:57.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:31:57.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T06:31:57.347 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T06:31:57.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T06:31:57.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T06:31:57.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T06:31:57.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T06:31:57.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T06:31:57.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T06:31:57.703 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T06:31:57.731 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T06:31:57.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T06:31:57.857 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:57.859 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T06:31:57.914 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T06:31:57.915 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T06:31:57.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T06:31:57.918 INFO:tasks.ceph.mon.a:Started 2023-01-25T06:31:57.918 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T06:31:57.919 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T06:31:57.920 INFO:tasks.ceph.mon.c:Started 2023-01-25T06:31:57.921 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T06:31:57.921 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T06:31:57.957 INFO:tasks.ceph.mon.b:Started 2023-01-25T06:31:57.958 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T06:31:57.958 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-25T06:31:57.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-25T06:31:57.963 INFO:tasks.ceph.mgr.y:Started 2023-01-25T06:31:57.964 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T06:31:57.964 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T06:31:57.966 INFO:tasks.ceph.mgr.x:Started 2023-01-25T06:31:57.967 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T06:31:57.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T06:32:04.363 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2023-01-25T06:32:04.379 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T06:32:04.379 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:32:04.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T06:32:04.408 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:32:04.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T06:32:04.476 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:32:04.477 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T06:32:04.544 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2023-01-25T06:32:04.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T06:32:04.612 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:32:04.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T06:32:04.640 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:32:04.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T06:32:04.709 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:32:04.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T06:32:04.777 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-01-25T06:32:04.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T06:32:04.844 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 35ab537b-0b4c-4201-8d19-ae2f26bb6bca 0 2023-01-25T06:32:05.098 INFO:teuthology.orchestra.run.smithi110.stdout:0 2023-01-25T06:32:05.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new f5427087-c30c-477b-8d8c-0d52faa69b7b 1 2023-01-25T06:32:05.178 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:05.176+0000 7f8a933b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T06:32:05.206 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:05.204+0000 7f9128e8a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T06:32:05.251 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:05.250+0000 7f8a933b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T06:32:05.279 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:05.278+0000 7f9128e8a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T06:32:05.325 INFO:teuthology.orchestra.run.smithi110.stdout:1 2023-01-25T06:32:05.337 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 3918efca-e289-4f0d-966d-f98eb66e12a9 2 2023-01-25T06:32:05.338 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:05.336+0000 7f8a933b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T06:32:05.373 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:05.371+0000 7f9128e8a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T06:32:05.529 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:05.527+0000 7f8a933b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T06:32:05.555 INFO:teuthology.orchestra.run.smithi110.stdout:2 2023-01-25T06:32:05.565 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:05.564+0000 7f9128e8a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T06:32:05.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 601c9ac3-dc46-4084-ba43-a74aacb3b21c 3 2023-01-25T06:32:05.780 INFO:teuthology.orchestra.run.smithi110.stdout:3 2023-01-25T06:32:05.793 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 814abf3f-0ec5-4f18-a7cb-35a080e3a176 4 2023-01-25T06:32:06.010 INFO:teuthology.orchestra.run.smithi110.stdout:4 2023-01-25T06:32:06.026 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 9bc191b3-128d-418d-90d2-136a77a61e07 5 2023-01-25T06:32:06.276 INFO:teuthology.orchestra.run.smithi110.stdout:5 2023-01-25T06:32:06.288 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new d5aa11ca-8ff2-4f20-8678-50a2d97f36f2 6 2023-01-25T06:32:06.347 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:06.345+0000 7f8a933b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T06:32:06.394 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:06.393+0000 7f9128e8a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T06:32:06.492 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:06.491+0000 7f8a933b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T06:32:06.503 INFO:teuthology.orchestra.run.smithi110.stdout:6 2023-01-25T06:32:06.515 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9 7 2023-01-25T06:32:06.541 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:06.540+0000 7f9128e8a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T06:32:06.721 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:06.720+0000 7f8a933b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T06:32:06.733 INFO:teuthology.orchestra.run.smithi110.stdout:7 2023-01-25T06:32:06.745 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T06:32:06.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T06:32:06.748 INFO:tasks.ceph.osd.0:Started 2023-01-25T06:32:06.749 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T06:32:06.749 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T06:32:06.752 INFO:tasks.ceph.osd.1:Started 2023-01-25T06:32:06.752 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T06:32:06.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T06:32:06.756 INFO:tasks.ceph.osd.2:Started 2023-01-25T06:32:06.757 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T06:32:06.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T06:32:06.759 INFO:tasks.ceph.osd.3:Started 2023-01-25T06:32:06.760 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T06:32:06.760 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T06:32:06.762 INFO:tasks.ceph.osd.4:Started 2023-01-25T06:32:06.763 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T06:32:06.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T06:32:06.765 INFO:tasks.ceph.osd.5:Started 2023-01-25T06:32:06.766 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T06:32:06.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T06:32:06.768 INFO:tasks.ceph.osd.6:Started 2023-01-25T06:32:06.769 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T06:32:06.769 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T06:32:06.771 INFO:tasks.ceph.osd.7:Started 2023-01-25T06:32:06.772 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:32:06.790 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:06.786+0000 7f9128e8a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T06:32:06.806 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:06.805+0000 7f8a933b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T06:32:06.877 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:06.875+0000 7f9128e8a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T06:32:06.965 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:06.964+0000 7f8a933b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:32:07.011 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:07.012 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"fsid":"b10aa3db-f868-4fb2-8582-d4662bf923a5","created":"2023-01-25T06:32:04.249805+0000","modified":"2023-01-25T06:32:06.730954+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T06:32:06.730954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"35ab537b-0b4c-4201-8d19-ae2f26bb6bca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f5427087-c30c-477b-8d8c-0d52faa69b7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3918efca-e289-4f0d-966d-f98eb66e12a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"601c9ac3-dc46-4084-ba43-a74aacb3b21c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"814abf3f-0ec5-4f18-a7cb-35a080e3a176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bc191b3-128d-418d-90d2-136a77a61e07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d5aa11ca-8ff2-4f20-8678-50a2d97f36f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:32:07.021 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.019+0000 7f9128e8a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:32:07.024 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T06:32:07.024 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T06:32:07.228 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.227+0000 7f8a933b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T06:32:07.301 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.300+0000 7f9128e8a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T06:32:07.368 INFO:tasks.ceph.osd.4.smithi110.stderr:2023-01-25T06:32:07.366+0000 7f8e642755c0 -1 Falling back to public interface 2023-01-25T06:32:07.368 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:07.366+0000 7f420a43e5c0 -1 Falling back to public interface 2023-01-25T06:32:07.369 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:07.366+0000 7f42017ec700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6800/0: (98) Address already in use 2023-01-25T06:32:07.369 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:07.366+0000 7f42017ec700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-01-25T06:32:07.369 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:07.367+0000 7f42017ec700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6804/0: (98) Address already in use 2023-01-25T06:32:07.370 INFO:tasks.ceph.osd.7.smithi110.stderr:2023-01-25T06:32:07.367+0000 7efdf00ae5c0 -1 Falling back to public interface 2023-01-25T06:32:07.376 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-25T06:32:07.374+0000 7f9a67ce75c0 -1 Falling back to public interface 2023-01-25T06:32:07.385 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-25T06:32:07.383+0000 7f37d16f45c0 -1 Falling back to public interface 2023-01-25T06:32:07.385 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-25T06:32:07.383+0000 7f9b896325c0 -1 Falling back to public interface 2023-01-25T06:32:07.385 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-25T06:32:07.383+0000 7f37c8aa2700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6804/0: (98) Address already in use 2023-01-25T06:32:07.390 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-25T06:32:07.388+0000 7f44a61145c0 -1 Falling back to public interface 2023-01-25T06:32:07.396 INFO:tasks.ceph.osd.6.smithi110.stderr:2023-01-25T06:32:07.395+0000 7f7d6c5bf5c0 -1 Falling back to public interface 2023-01-25T06:32:07.415 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.413+0000 7f8a933b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T06:32:07.494 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.492+0000 7f9128e8a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T06:32:07.555 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.553+0000 7f8a933b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T06:32:07.639 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.637+0000 7f9128e8a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T06:32:07.671 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.670+0000 7f8a933b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T06:32:07.760 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.758+0000 7f9128e8a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T06:32:07.810 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.809+0000 7f8a933b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:32:07.830 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T06:32:07.889 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:07.887+0000 7f8a933b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T06:32:07.907 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.906+0000 7f9128e8a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T06:32:07.994 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:07.992+0000 7f9128e8a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T06:32:08.029 INFO:teuthology.misc.health.smithi062.stdout: 2023-01-25T06:32:08.029 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":10,"fsid":"b10aa3db-f868-4fb2-8582-d4662bf923a5","created":"2023-01-25T06:32:04.249805+0000","modified":"2023-01-25T06:32:06.730954+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T06:32:06.730954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"35ab537b-0b4c-4201-8d19-ae2f26bb6bca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f5427087-c30c-477b-8d8c-0d52faa69b7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3918efca-e289-4f0d-966d-f98eb66e12a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"601c9ac3-dc46-4084-ba43-a74aacb3b21c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"814abf3f-0ec5-4f18-a7cb-35a080e3a176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9bc191b3-128d-418d-90d2-136a77a61e07","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d5aa11ca-8ff2-4f20-8678-50a2d97f36f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:32:08.041 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-25T06:32:08.108 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:08.106+0000 7f8a933b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T06:32:08.184 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:08.183+0000 7f8a933b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T06:32:08.221 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:08.220+0000 7f9128e8a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T06:32:08.302 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:08.300+0000 7f9128e8a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T06:32:08.797 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:08.796+0000 7f8a933b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T06:32:08.946 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:08.945+0000 7f9128e8a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T06:32:09.050 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.048+0000 7f8a933b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T06:32:09.125 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.123+0000 7f8a933b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T06:32:09.205 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.204+0000 7f8a933b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T06:32:09.211 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.209+0000 7f9128e8a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T06:32:09.288 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.287+0000 7f9128e8a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T06:32:09.373 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.372+0000 7f9128e8a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T06:32:09.388 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.387+0000 7f8a933b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T06:32:09.459 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.458+0000 7f8a933b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T06:32:09.563 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.561+0000 7f9128e8a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T06:32:09.611 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:09.609+0000 7f8a933b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T06:32:09.638 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.637+0000 7f9128e8a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T06:32:09.799 INFO:tasks.ceph.mgr.y.smithi062.stderr:2023-01-25T06:32:09.797+0000 7f9128e8a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T06:32:10.635 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:10.634+0000 7f8a4cef6700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T06:32:10.638 INFO:tasks.ceph.mgr.x.smithi110.stderr:2023-01-25T06:32:10.637+0000 7f8a4cef6700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T06:32:10.864 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-25T06:32:10.863+0000 7f9a67ce75c0 -1 osd.0 0 log_to_monitors true 2023-01-25T06:32:10.884 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:10.883+0000 7f420a43e5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T06:32:10.891 INFO:tasks.ceph.osd.4.smithi110.stderr:2023-01-25T06:32:10.890+0000 7f8e642755c0 -1 osd.4 0 log_to_monitors true 2023-01-25T06:32:10.896 INFO:tasks.ceph.osd.7.smithi110.stderr:2023-01-25T06:32:10.895+0000 7efdf00ae5c0 -1 osd.7 0 log_to_monitors true 2023-01-25T06:32:10.900 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-25T06:32:10.898+0000 7f37d16f45c0 -1 osd.2 0 log_to_monitors true 2023-01-25T06:32:10.917 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-25T06:32:10.915+0000 7f9b896325c0 -1 osd.3 0 log_to_monitors true 2023-01-25T06:32:10.918 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-25T06:32:10.917+0000 7f44a61145c0 -1 osd.1 0 log_to_monitors true 2023-01-25T06:32:10.923 INFO:tasks.ceph.osd.6.smithi110.stderr:2023-01-25T06:32:10.922+0000 7f7d6c5bf5c0 -1 osd.6 0 log_to_monitors true 2023-01-25T06:32:12.640 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-25T06:32:12.639+0000 7f37c529b700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T06:32:12.642 INFO:tasks.ceph.osd.6.smithi110.stderr:2023-01-25T06:32:12.641+0000 7f7d60166700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T06:32:12.643 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-25T06:32:12.642+0000 7f9a5b88e700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T06:32:12.646 INFO:tasks.ceph.osd.4.smithi110.stderr:2023-01-25T06:32:12.645+0000 7f8e57e1c700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T06:32:12.646 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:12.645+0000 7f41fdfe5700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T06:32:12.647 INFO:tasks.ceph.osd.7.smithi110.stderr:2023-01-25T06:32:12.645+0000 7efde3c55700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T06:32:12.647 INFO:tasks.ceph.osd.2.smithi062.stderr:2023-01-25T06:32:12.645+0000 7f37bd676700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.648 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-25T06:32:12.646+0000 7f9b7d1d9700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T06:32:12.648 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-25T06:32:12.646+0000 7f4499cbb700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T06:32:12.651 INFO:tasks.ceph.osd.4.smithi110.stderr:2023-01-25T06:32:12.650+0000 7f8e501f7700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.651 INFO:tasks.ceph.osd.5.smithi110.stderr:2023-01-25T06:32:12.650+0000 7f41f63c0700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.652 INFO:tasks.ceph.osd.7.smithi110.stderr:2023-01-25T06:32:12.650+0000 7efddc030700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.652 INFO:tasks.ceph.osd.6.smithi110.stderr:2023-01-25T06:32:12.651+0000 7f7d58541700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.653 INFO:tasks.ceph.osd.0.smithi062.stderr:2023-01-25T06:32:12.651+0000 7f9a53c69700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.653 INFO:tasks.ceph.osd.3.smithi062.stderr:2023-01-25T06:32:12.652+0000 7f9b755b4700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:12.654 INFO:tasks.ceph.osd.1.smithi062.stderr:2023-01-25T06:32:12.652+0000 7f4492096700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T06:32:14.848 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T06:32:15.187 INFO:teuthology.misc.health.smithi062.stdout: 2023-01-25T06:32:15.187 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":14,"fsid":"b10aa3db-f868-4fb2-8582-d4662bf923a5","created":"2023-01-25T06:32:04.249805+0000","modified":"2023-01-25T06:32:14.655937+0000","last_up_change":"2023-01-25T06:32:13.649013+0000","last_in_change":"2023-01-25T06:32:06.730954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T06:32:13.692575+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"35ab537b-0b4c-4201-8d19-ae2f26bb6bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6802","nonce":106361}]},"public_addr":"172.21.15.62:6800/106361","cluster_addr":"172.21.15.62:6801/106361","heartbeat_back_addr":"172.21.15.62:6803/106361","heartbeat_front_addr":"172.21.15.62:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"f5427087-c30c-477b-8d8c-0d52faa69b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6812","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6813","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6815","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6814","nonce":106363}]},"public_addr":"172.21.15.62:6812/106363","cluster_addr":"172.21.15.62:6813/106363","heartbeat_back_addr":"172.21.15.62:6815/106363","heartbeat_front_addr":"172.21.15.62:6814/106363","state":["exists","up"]},{"osd":2,"uuid":"3918efca-e289-4f0d-966d-f98eb66e12a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6805","nonce":106364}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6807","nonce":106364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6811","nonce":106364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6809","nonce":106364}]},"public_addr":"172.21.15.62:6805/106364","cluster_addr":"172.21.15.62:6807/106364","heartbeat_back_addr":"172.21.15.62:6811/106364","heartbeat_front_addr":"172.21.15.62:6809/106364","state":["exists","up"]},{"osd":3,"uuid":"601c9ac3-dc46-4084-ba43-a74aacb3b21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6804","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6806","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6810","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6808","nonce":106362}]},"public_addr":"172.21.15.62:6804/106362","cluster_addr":"172.21.15.62:6806/106362","heartbeat_back_addr":"172.21.15.62:6810/106362","heartbeat_front_addr":"172.21.15.62:6808/106362","state":["exists","up"]},{"osd":4,"uuid":"814abf3f-0ec5-4f18-a7cb-35a080e3a176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6800","nonce":106280}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6801","nonce":106280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6804","nonce":106280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6803","nonce":106280}]},"public_addr":"172.21.15.110:6800/106280","cluster_addr":"172.21.15.110:6801/106280","heartbeat_back_addr":"172.21.15.110:6804/106280","heartbeat_front_addr":"172.21.15.110:6803/106280","state":["exists","up"]},{"osd":5,"uuid":"9bc191b3-128d-418d-90d2-136a77a61e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6802","nonce":106279}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6805","nonce":106279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6807","nonce":106279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6806","nonce":106279}]},"public_addr":"172.21.15.110:6802/106279","cluster_addr":"172.21.15.110:6805/106279","heartbeat_back_addr":"172.21.15.110:6807/106279","heartbeat_front_addr":"172.21.15.110:6806/106279","state":["exists","up"]},{"osd":6,"uuid":"d5aa11ca-8ff2-4f20-8678-50a2d97f36f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6812","nonce":106324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6813","nonce":106324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6815","nonce":106324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6814","nonce":106324}]},"public_addr":"172.21.15.110:6812/106324","cluster_addr":"172.21.15.110:6813/106324","heartbeat_back_addr":"172.21.15.110:6815/106324","heartbeat_front_addr":"172.21.15.110:6814/106324","state":["exists","up"]},{"osd":7,"uuid":"1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6808","nonce":106281}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6809","nonce":106281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6811","nonce":106281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6810","nonce":106281}]},"public_addr":"172.21.15.110:6808/106281","cluster_addr":"172.21.15.110:6809/106281","heartbeat_back_addr":"172.21.15.110:6811/106281","heartbeat_front_addr":"172.21.15.110:6810/106281","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.911247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.920661+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.898603+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.903479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.865019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.844666+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.923181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.850904+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:32:15.200 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-25T06:32:15.200 INFO:tasks.ceph:Creating RBD pool 2023-01-25T06:32:15.201 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T06:32:15.672 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'rbd' created 2023-01-25T06:32:15.684 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T06:32:16.690 INFO:teuthology.orchestra.run.smithi062.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T06:32:16.702 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T06:32:16.703 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T06:32:16.703 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T06:32:16.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T06:32:16.704 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T06:32:17.096 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:17.111 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":4,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6816","nonce":105616}]},"active_addr":"172.21.15.110:6816/105616","active_change":"2023-01-25T06:32:09.624199+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3440090443}]},{"addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1875066860}]},{"addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1893160111}]}]}} 2023-01-25T06:32:17.115 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T06:32:17.115 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T06:32:17.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:32:17.443 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:17.444 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"b10aa3db-f868-4fb2-8582-d4662bf923a5","created":"2023-01-25T06:32:04.249805+0000","modified":"2023-01-25T06:32:16.681518+0000","last_up_change":"2023-01-25T06:32:13.649013+0000","last_in_change":"2023-01-25T06:32:06.730954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T06:32:13.692575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T06:32:15.558294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"35ab537b-0b4c-4201-8d19-ae2f26bb6bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6802","nonce":106361}]},"public_addr":"172.21.15.62:6800/106361","cluster_addr":"172.21.15.62:6801/106361","heartbeat_back_addr":"172.21.15.62:6803/106361","heartbeat_front_addr":"172.21.15.62:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"f5427087-c30c-477b-8d8c-0d52faa69b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6812","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6813","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6815","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6814","nonce":106363}]},"public_addr":"172.21.15.62:6812/106363","cluster_addr":"172.21.15.62:6813/106363","heartbeat_back_addr":"172.21.15.62:6815/106363","heartbeat_front_addr":"172.21.15.62:6814/106363","state":["exists","up"]},{"osd":2,"uuid":"3918efca-e289-4f0d-966d-f98eb66e12a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6805","nonce":106364}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6807","nonce":106364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6811","nonce":106364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6809","nonce":106364}]},"public_addr":"172.21.15.62:6805/106364","cluster_addr":"172.21.15.62:6807/106364","heartbeat_back_addr":"172.21.15.62:6811/106364","heartbeat_front_addr":"172.21.15.62:6809/106364","state":["exists","up"]},{"osd":3,"uuid":"601c9ac3-dc46-4084-ba43-a74aacb3b21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6804","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6806","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6810","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6808","nonce":106362}]},"public_addr":"172.21.15.62:6804/106362","cluster_addr":"172.21.15.62:6806/106362","heartbeat_back_addr":"172.21.15.62:6810/106362","heartbeat_front_addr":"172.21.15.62:6808/106362","state":["exists","up"]},{"osd":4,"uuid":"814abf3f-0ec5-4f18-a7cb-35a080e3a176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6800","nonce":106280}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6801","nonce":106280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6804","nonce":106280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6803","nonce":106280}]},"public_addr":"172.21.15.110:6800/106280","cluster_addr":"172.21.15.110:6801/106280","heartbeat_back_addr":"172.21.15.110:6804/106280","heartbeat_front_addr":"172.21.15.110:6803/106280","state":["exists","up"]},{"osd":5,"uuid":"9bc191b3-128d-418d-90d2-136a77a61e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6802","nonce":106279}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6805","nonce":106279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6807","nonce":106279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6806","nonce":106279}]},"public_addr":"172.21.15.110:6802/106279","cluster_addr":"172.21.15.110:6805/106279","heartbeat_back_addr":"172.21.15.110:6807/106279","heartbeat_front_addr":"172.21.15.110:6806/106279","state":["exists","up"]},{"osd":6,"uuid":"d5aa11ca-8ff2-4f20-8678-50a2d97f36f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6812","nonce":106324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6813","nonce":106324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6815","nonce":106324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6814","nonce":106324}]},"public_addr":"172.21.15.110:6812/106324","cluster_addr":"172.21.15.110:6813/106324","heartbeat_back_addr":"172.21.15.110:6815/106324","heartbeat_front_addr":"172.21.15.110:6814/106324","state":["exists","up"]},{"osd":7,"uuid":"1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6808","nonce":106281}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6809","nonce":106281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6811","nonce":106281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6810","nonce":106281}]},"public_addr":"172.21.15.110:6808/106281","cluster_addr":"172.21.15.110:6809/106281","heartbeat_back_addr":"172.21.15.110:6811/106281","heartbeat_front_addr":"172.21.15.110:6810/106281","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.911247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.920661+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.898603+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.903479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.865019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.844666+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.923181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.850904+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:32:17.457 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T06:32:17.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T06:32:17.792 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:17.793 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"fsid":"b10aa3db-f868-4fb2-8582-d4662bf923a5","created":"2023-01-25T06:32:04.249805+0000","modified":"2023-01-25T06:32:17.686657+0000","last_up_change":"2023-01-25T06:32:13.649013+0000","last_in_change":"2023-01-25T06:32:06.730954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-25T06:32:13.692575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-25T06:32:15.558294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"35ab537b-0b4c-4201-8d19-ae2f26bb6bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6802","nonce":106361}]},"public_addr":"172.21.15.62:6800/106361","cluster_addr":"172.21.15.62:6801/106361","heartbeat_back_addr":"172.21.15.62:6803/106361","heartbeat_front_addr":"172.21.15.62:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"f5427087-c30c-477b-8d8c-0d52faa69b7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6812","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6813","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6815","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6814","nonce":106363}]},"public_addr":"172.21.15.62:6812/106363","cluster_addr":"172.21.15.62:6813/106363","heartbeat_back_addr":"172.21.15.62:6815/106363","heartbeat_front_addr":"172.21.15.62:6814/106363","state":["exists","up"]},{"osd":2,"uuid":"3918efca-e289-4f0d-966d-f98eb66e12a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6805","nonce":106364}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6807","nonce":106364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6811","nonce":106364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6809","nonce":106364}]},"public_addr":"172.21.15.62:6805/106364","cluster_addr":"172.21.15.62:6807/106364","heartbeat_back_addr":"172.21.15.62:6811/106364","heartbeat_front_addr":"172.21.15.62:6809/106364","state":["exists","up"]},{"osd":3,"uuid":"601c9ac3-dc46-4084-ba43-a74aacb3b21c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6804","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6806","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6810","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.62:6808","nonce":106362}]},"public_addr":"172.21.15.62:6804/106362","cluster_addr":"172.21.15.62:6806/106362","heartbeat_back_addr":"172.21.15.62:6810/106362","heartbeat_front_addr":"172.21.15.62:6808/106362","state":["exists","up"]},{"osd":4,"uuid":"814abf3f-0ec5-4f18-a7cb-35a080e3a176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6800","nonce":106280}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6801","nonce":106280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6804","nonce":106280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6803","nonce":106280}]},"public_addr":"172.21.15.110:6800/106280","cluster_addr":"172.21.15.110:6801/106280","heartbeat_back_addr":"172.21.15.110:6804/106280","heartbeat_front_addr":"172.21.15.110:6803/106280","state":["exists","up"]},{"osd":5,"uuid":"9bc191b3-128d-418d-90d2-136a77a61e07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6802","nonce":106279}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6805","nonce":106279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6807","nonce":106279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6806","nonce":106279}]},"public_addr":"172.21.15.110:6802/106279","cluster_addr":"172.21.15.110:6805/106279","heartbeat_back_addr":"172.21.15.110:6807/106279","heartbeat_front_addr":"172.21.15.110:6806/106279","state":["exists","up"]},{"osd":6,"uuid":"d5aa11ca-8ff2-4f20-8678-50a2d97f36f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6812","nonce":106324}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6813","nonce":106324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6815","nonce":106324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6814","nonce":106324}]},"public_addr":"172.21.15.110:6812/106324","cluster_addr":"172.21.15.110:6813/106324","heartbeat_back_addr":"172.21.15.110:6815/106324","heartbeat_front_addr":"172.21.15.110:6814/106324","state":["exists","up"]},{"osd":7,"uuid":"1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6808","nonce":106281}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6809","nonce":106281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6811","nonce":106281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.110:6810","nonce":106281}]},"public_addr":"172.21.15.110:6808/106281","cluster_addr":"172.21.15.110:6809/106281","heartbeat_back_addr":"172.21.15.110:6811/106281","heartbeat_front_addr":"172.21.15.110:6810/106281","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.911247+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.920661+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.898603+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.903479+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.865019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.844666+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.923181+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T06:32:11.850904+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T06:32:17.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T06:32:17.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T06:32:17.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T06:32:17.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T06:32:17.809 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T06:32:17.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T06:32:17.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T06:32:17.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T06:32:18.111 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T06:32:18.112 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:18.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T06:32:18.114 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T06:32:18.116 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T06:32:18.120 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.121 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T06:32:18.125 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:18.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T06:32:18.133 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.133 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T06:32:18.134 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:18.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T06:32:18.740 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-25T06:32:18.743 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-25T06:32:18.757 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-25T06:32:18.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-01-25T06:32:18.768 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.770 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-25T06:32:18.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-01-25T06:32:18.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2023-01-25T06:32:18.792 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2023-01-25T06:32:18.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2023-01-25T06:32:18.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-25T06:32:18.823 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:18.826 INFO:teuthology.orchestra.run.smithi062.stdout:55834574849 2023-01-25T06:32:18.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-25T06:32:18.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.6 2023-01-25T06:32:19.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T06:32:19.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T06:32:19.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T06:32:19.794 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T06:32:19.795 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T06:32:19.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T06:32:19.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T06:32:19.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T06:32:20.412 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:20.425 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:20.430 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:20.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.2 2023-01-25T06:32:20.433 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.439 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:20.450 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:20.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.0 2023-01-25T06:32:20.451 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-25T06:32:20.459 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.1 2023-01-25T06:32:20.469 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.4 2023-01-25T06:32:20.474 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.482 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:20.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-25T06:32:20.496 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.503 INFO:teuthology.orchestra.run.smithi062.stdout:55834574850 2023-01-25T06:32:20.509 INFO:teuthology.orchestra.run.smithi062.stdout:55834574851 2023-01-25T06:32:20.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.6 2023-01-25T06:32:20.518 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-25T06:32:20.525 DEBUG:teuthology.parallel:result is None 2023-01-25T06:32:20.525 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T06:32:20.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:32:20.890 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:20.892 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-25T06:32:20.905 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T06:32:19.629795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44348,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":41152,"kb_avail":754930372,"statfs":{"total":773094113280,"available":773048700928,"internally_reserved":0,"allocated":2752512,"data_stored":1942420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42139648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.690524+0000","last_change":"2023-01-25T06:32:16.692677+0000","last_active":"2023-01-25T06:32:17.690524+0000","last_peered":"2023-01-25T06:32:17.690524+0000","last_clean":"2023-01-25T06:32:17.690524+0000","last_became_active":"2023-01-25T06:32:16.692404+0000","last_became_peered":"2023-01-25T06:32:16.692404+0000","last_unstale":"2023-01-25T06:32:17.690524+0000","last_undegraded":"2023-01-25T06:32:17.690524+0000","last_fullsized":"2023-01-25T06:32:17.690524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:20.379707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.938070+0000","last_change":"2023-01-25T06:32:16.688495+0000","last_active":"2023-01-25T06:32:17.938070+0000","last_peered":"2023-01-25T06:32:17.938070+0000","last_clean":"2023-01-25T06:32:17.938070+0000","last_became_active":"2023-01-25T06:32:16.688141+0000","last_became_peered":"2023-01-25T06:32:16.688141+0000","last_unstale":"2023-01-25T06:32:17.938070+0000","last_undegraded":"2023-01-25T06:32:17.938070+0000","last_fullsized":"2023-01-25T06:32:17.938070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:16.268500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927167+0000","last_change":"2023-01-25T06:32:16.756852+0000","last_active":"2023-01-25T06:32:17.927167+0000","last_peered":"2023-01-25T06:32:17.927167+0000","last_clean":"2023-01-25T06:32:17.927167+0000","last_became_active":"2023-01-25T06:32:16.756523+0000","last_became_peered":"2023-01-25T06:32:16.756523+0000","last_unstale":"2023-01-25T06:32:17.927167+0000","last_undegraded":"2023-01-25T06:32:17.927167+0000","last_fullsized":"2023-01-25T06:32:17.927167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:35.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.756849+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.756514+0000","last_became_peered":"2023-01-25T06:32:16.756514+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:26.282967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927169+0000","last_change":"2023-01-25T06:32:15.673719+0000","last_active":"2023-01-25T06:32:17.927169+0000","last_peered":"2023-01-25T06:32:17.927169+0000","last_clean":"2023-01-25T06:32:17.927169+0000","last_became_active":"2023-01-25T06:32:15.673394+0000","last_became_peered":"2023-01-25T06:32:15.673394+0000","last_unstale":"2023-01-25T06:32:17.927169+0000","last_undegraded":"2023-01-25T06:32:17.927169+0000","last_fullsized":"2023-01-25T06:32:17.927169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_clean_scrub_stamp":"2023-01-25T06:32:14.655937+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:22.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927082+0000","last_change":"2023-01-25T06:32:16.692900+0000","last_active":"2023-01-25T06:32:17.927082+0000","last_peered":"2023-01-25T06:32:17.927082+0000","last_clean":"2023-01-25T06:32:17.927082+0000","last_became_active":"2023-01-25T06:32:16.692609+0000","last_became_peered":"2023-01-25T06:32:16.692609+0000","last_unstale":"2023-01-25T06:32:17.927082+0000","last_undegraded":"2023-01-25T06:32:17.927082+0000","last_fullsized":"2023-01-25T06:32:17.927082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:44.684692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.960305+0000","last_change":"2023-01-25T06:32:16.756988+0000","last_active":"2023-01-25T06:32:17.960305+0000","last_peered":"2023-01-25T06:32:17.960305+0000","last_clean":"2023-01-25T06:32:17.960305+0000","last_became_active":"2023-01-25T06:32:16.756599+0000","last_became_peered":"2023-01-25T06:32:16.756599+0000","last_unstale":"2023-01-25T06:32:17.960305+0000","last_undegraded":"2023-01-25T06:32:17.960305+0000","last_fullsized":"2023-01-25T06:32:17.960305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:23.668972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.692157+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.691857+0000","last_became_peered":"2023-01-25T06:32:16.691857+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:43.658061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.937951+0000","last_change":"2023-01-25T06:32:16.756859+0000","last_active":"2023-01-25T06:32:17.937951+0000","last_peered":"2023-01-25T06:32:17.937951+0000","last_clean":"2023-01-25T06:32:17.937951+0000","last_became_active":"2023-01-25T06:32:16.756503+0000","last_became_peered":"2023-01-25T06:32:16.756503+0000","last_unstale":"2023-01-25T06:32:17.937951+0000","last_undegraded":"2023-01-25T06:32:17.937951+0000","last_fullsized":"2023-01-25T06:32:17.937951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:42.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4190000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6369999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":139264,"data_stored":41562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6548,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365292,"statfs":{"total":96636764160,"available":96630059008,"internally_reserved":0,"allocated":671744,"data_stored":570490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T06:32:20.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:32:21.233 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:21.235 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-25T06:32:21.246 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T06:32:19.629795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44348,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":41152,"kb_avail":754930372,"statfs":{"total":773094113280,"available":773048700928,"internally_reserved":0,"allocated":2752512,"data_stored":1942420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42139648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.690524+0000","last_change":"2023-01-25T06:32:16.692677+0000","last_active":"2023-01-25T06:32:17.690524+0000","last_peered":"2023-01-25T06:32:17.690524+0000","last_clean":"2023-01-25T06:32:17.690524+0000","last_became_active":"2023-01-25T06:32:16.692404+0000","last_became_peered":"2023-01-25T06:32:16.692404+0000","last_unstale":"2023-01-25T06:32:17.690524+0000","last_undegraded":"2023-01-25T06:32:17.690524+0000","last_fullsized":"2023-01-25T06:32:17.690524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:20.379707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.938070+0000","last_change":"2023-01-25T06:32:16.688495+0000","last_active":"2023-01-25T06:32:17.938070+0000","last_peered":"2023-01-25T06:32:17.938070+0000","last_clean":"2023-01-25T06:32:17.938070+0000","last_became_active":"2023-01-25T06:32:16.688141+0000","last_became_peered":"2023-01-25T06:32:16.688141+0000","last_unstale":"2023-01-25T06:32:17.938070+0000","last_undegraded":"2023-01-25T06:32:17.938070+0000","last_fullsized":"2023-01-25T06:32:17.938070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:16.268500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927167+0000","last_change":"2023-01-25T06:32:16.756852+0000","last_active":"2023-01-25T06:32:17.927167+0000","last_peered":"2023-01-25T06:32:17.927167+0000","last_clean":"2023-01-25T06:32:17.927167+0000","last_became_active":"2023-01-25T06:32:16.756523+0000","last_became_peered":"2023-01-25T06:32:16.756523+0000","last_unstale":"2023-01-25T06:32:17.927167+0000","last_undegraded":"2023-01-25T06:32:17.927167+0000","last_fullsized":"2023-01-25T06:32:17.927167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:35.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.756849+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.756514+0000","last_became_peered":"2023-01-25T06:32:16.756514+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:26.282967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927169+0000","last_change":"2023-01-25T06:32:15.673719+0000","last_active":"2023-01-25T06:32:17.927169+0000","last_peered":"2023-01-25T06:32:17.927169+0000","last_clean":"2023-01-25T06:32:17.927169+0000","last_became_active":"2023-01-25T06:32:15.673394+0000","last_became_peered":"2023-01-25T06:32:15.673394+0000","last_unstale":"2023-01-25T06:32:17.927169+0000","last_undegraded":"2023-01-25T06:32:17.927169+0000","last_fullsized":"2023-01-25T06:32:17.927169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_clean_scrub_stamp":"2023-01-25T06:32:14.655937+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:22.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927082+0000","last_change":"2023-01-25T06:32:16.692900+0000","last_active":"2023-01-25T06:32:17.927082+0000","last_peered":"2023-01-25T06:32:17.927082+0000","last_clean":"2023-01-25T06:32:17.927082+0000","last_became_active":"2023-01-25T06:32:16.692609+0000","last_became_peered":"2023-01-25T06:32:16.692609+0000","last_unstale":"2023-01-25T06:32:17.927082+0000","last_undegraded":"2023-01-25T06:32:17.927082+0000","last_fullsized":"2023-01-25T06:32:17.927082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:44.684692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.960305+0000","last_change":"2023-01-25T06:32:16.756988+0000","last_active":"2023-01-25T06:32:17.960305+0000","last_peered":"2023-01-25T06:32:17.960305+0000","last_clean":"2023-01-25T06:32:17.960305+0000","last_became_active":"2023-01-25T06:32:16.756599+0000","last_became_peered":"2023-01-25T06:32:16.756599+0000","last_unstale":"2023-01-25T06:32:17.960305+0000","last_undegraded":"2023-01-25T06:32:17.960305+0000","last_fullsized":"2023-01-25T06:32:17.960305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:23.668972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.692157+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.691857+0000","last_became_peered":"2023-01-25T06:32:16.691857+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:43.658061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.937951+0000","last_change":"2023-01-25T06:32:16.756859+0000","last_active":"2023-01-25T06:32:17.937951+0000","last_peered":"2023-01-25T06:32:17.937951+0000","last_clean":"2023-01-25T06:32:17.937951+0000","last_became_active":"2023-01-25T06:32:16.756503+0000","last_became_peered":"2023-01-25T06:32:16.756503+0000","last_unstale":"2023-01-25T06:32:17.937951+0000","last_undegraded":"2023-01-25T06:32:17.937951+0000","last_fullsized":"2023-01-25T06:32:17.937951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:42.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4190000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6369999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5136,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366704,"statfs":{"total":96636764160,"available":96631504896,"internally_reserved":0,"allocated":139264,"data_stored":41562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6548,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365292,"statfs":{"total":96636764160,"available":96630059008,"internally_reserved":0,"allocated":671744,"data_stored":570490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5072,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366768,"statfs":{"total":96636764160,"available":96631570432,"internally_reserved":0,"allocated":139264,"data_stored":41562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T06:32:21.248 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T06:32:21.248 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T06:32:21.248 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T06:32:21.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T06:32:21.621 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:21.622 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T06:32:21.636 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T06:32:21.636 INFO:teuthology.run_tasks:Running task thrashosds... 2023-01-25T06:32:21.646 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-01-25T06:32:21.646 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri2-testing-2023-01-23-0928'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore queue throttle high multiple': 2, 'filestore queue throttle max multiple': 10, 'journal throttle high multiple': 2, 'journal throttle max multiple': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 9, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'must scrub before tier agent can activate'], 'mon_bind_msgr2': False, 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-yuri2-testing-2023-01-23-0928', 'sha1': '61996d73ab738798e4ee0055c86457f5c459da4a'}} 2023-01-25T06:32:21.647 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 3, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2023-01-25T06:32:21.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-01-25T06:32:21.996 INFO:tasks.thrashosds:Beginning thrashosds... 2023-01-25T06:32:21.997 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T06:32:21.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:32:22.327 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:22.330 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-25T06:32:22.341 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T06:32:21.630198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44436,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":41216,"kb_avail":754930284,"statfs":{"total":773094113280,"available":773048610816,"internally_reserved":0,"allocated":2777088,"data_stored":1956304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42205184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000403"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.690524+0000","last_change":"2023-01-25T06:32:16.692677+0000","last_active":"2023-01-25T06:32:17.690524+0000","last_peered":"2023-01-25T06:32:17.690524+0000","last_clean":"2023-01-25T06:32:17.690524+0000","last_became_active":"2023-01-25T06:32:16.692404+0000","last_became_peered":"2023-01-25T06:32:16.692404+0000","last_unstale":"2023-01-25T06:32:17.690524+0000","last_undegraded":"2023-01-25T06:32:17.690524+0000","last_fullsized":"2023-01-25T06:32:17.690524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:20.379707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.938070+0000","last_change":"2023-01-25T06:32:16.688495+0000","last_active":"2023-01-25T06:32:17.938070+0000","last_peered":"2023-01-25T06:32:17.938070+0000","last_clean":"2023-01-25T06:32:17.938070+0000","last_became_active":"2023-01-25T06:32:16.688141+0000","last_became_peered":"2023-01-25T06:32:16.688141+0000","last_unstale":"2023-01-25T06:32:17.938070+0000","last_undegraded":"2023-01-25T06:32:17.938070+0000","last_fullsized":"2023-01-25T06:32:17.938070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:16.268500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927167+0000","last_change":"2023-01-25T06:32:16.756852+0000","last_active":"2023-01-25T06:32:17.927167+0000","last_peered":"2023-01-25T06:32:17.927167+0000","last_clean":"2023-01-25T06:32:17.927167+0000","last_became_active":"2023-01-25T06:32:16.756523+0000","last_became_peered":"2023-01-25T06:32:16.756523+0000","last_unstale":"2023-01-25T06:32:17.927167+0000","last_undegraded":"2023-01-25T06:32:17.927167+0000","last_fullsized":"2023-01-25T06:32:17.927167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:35.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.756849+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.756514+0000","last_became_peered":"2023-01-25T06:32:16.756514+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:26.282967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927169+0000","last_change":"2023-01-25T06:32:15.673719+0000","last_active":"2023-01-25T06:32:17.927169+0000","last_peered":"2023-01-25T06:32:17.927169+0000","last_clean":"2023-01-25T06:32:17.927169+0000","last_became_active":"2023-01-25T06:32:15.673394+0000","last_became_peered":"2023-01-25T06:32:15.673394+0000","last_unstale":"2023-01-25T06:32:17.927169+0000","last_undegraded":"2023-01-25T06:32:17.927169+0000","last_fullsized":"2023-01-25T06:32:17.927169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_clean_scrub_stamp":"2023-01-25T06:32:14.655937+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:22.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927082+0000","last_change":"2023-01-25T06:32:16.692900+0000","last_active":"2023-01-25T06:32:17.927082+0000","last_peered":"2023-01-25T06:32:17.927082+0000","last_clean":"2023-01-25T06:32:17.927082+0000","last_became_active":"2023-01-25T06:32:16.692609+0000","last_became_peered":"2023-01-25T06:32:16.692609+0000","last_unstale":"2023-01-25T06:32:17.927082+0000","last_undegraded":"2023-01-25T06:32:17.927082+0000","last_fullsized":"2023-01-25T06:32:17.927082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:44.684692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.960305+0000","last_change":"2023-01-25T06:32:16.756988+0000","last_active":"2023-01-25T06:32:17.960305+0000","last_peered":"2023-01-25T06:32:17.960305+0000","last_clean":"2023-01-25T06:32:17.960305+0000","last_became_active":"2023-01-25T06:32:16.756599+0000","last_became_peered":"2023-01-25T06:32:16.756599+0000","last_unstale":"2023-01-25T06:32:17.960305+0000","last_undegraded":"2023-01-25T06:32:17.960305+0000","last_fullsized":"2023-01-25T06:32:17.960305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:23.668972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.692157+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.691857+0000","last_became_peered":"2023-01-25T06:32:16.691857+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:43.658061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.937951+0000","last_change":"2023-01-25T06:32:16.756859+0000","last_active":"2023-01-25T06:32:17.937951+0000","last_peered":"2023-01-25T06:32:17.937951+0000","last_clean":"2023-01-25T06:32:17.937951+0000","last_became_active":"2023-01-25T06:32:16.756503+0000","last_became_peered":"2023-01-25T06:32:16.756503+0000","last_unstale":"2023-01-25T06:32:17.937951+0000","last_undegraded":"2023-01-25T06:32:17.937951+0000","last_fullsized":"2023-01-25T06:32:17.937951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:42.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4190000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6369999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6556,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365284,"statfs":{"total":96636764160,"available":96630050816,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.9339999999999993}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.9860000000000007}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9619999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1470000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T06:32:22.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T06:32:22.667 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-01-25T06:32:22.669 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2023-01-25T06:32:22.680 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2023-01-25T06:32:21.630198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":44436,"kb_used_data":2712,"kb_used_omap":0,"kb_used_meta":41216,"kb_avail":754930284,"statfs":{"total":773094113280,"available":773048610816,"internally_reserved":0,"allocated":2777088,"data_stored":1956304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":42205184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000403"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.690524+0000","last_change":"2023-01-25T06:32:16.692677+0000","last_active":"2023-01-25T06:32:17.690524+0000","last_peered":"2023-01-25T06:32:17.690524+0000","last_clean":"2023-01-25T06:32:17.690524+0000","last_became_active":"2023-01-25T06:32:16.692404+0000","last_became_peered":"2023-01-25T06:32:16.692404+0000","last_unstale":"2023-01-25T06:32:17.690524+0000","last_undegraded":"2023-01-25T06:32:17.690524+0000","last_fullsized":"2023-01-25T06:32:17.690524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:20.379707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.938070+0000","last_change":"2023-01-25T06:32:16.688495+0000","last_active":"2023-01-25T06:32:17.938070+0000","last_peered":"2023-01-25T06:32:17.938070+0000","last_clean":"2023-01-25T06:32:17.938070+0000","last_became_active":"2023-01-25T06:32:16.688141+0000","last_became_peered":"2023-01-25T06:32:16.688141+0000","last_unstale":"2023-01-25T06:32:17.938070+0000","last_undegraded":"2023-01-25T06:32:17.938070+0000","last_fullsized":"2023-01-25T06:32:17.938070+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:16.268500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927167+0000","last_change":"2023-01-25T06:32:16.756852+0000","last_active":"2023-01-25T06:32:17.927167+0000","last_peered":"2023-01-25T06:32:17.927167+0000","last_clean":"2023-01-25T06:32:17.927167+0000","last_became_active":"2023-01-25T06:32:16.756523+0000","last_became_peered":"2023-01-25T06:32:16.756523+0000","last_unstale":"2023-01-25T06:32:17.927167+0000","last_undegraded":"2023-01-25T06:32:17.927167+0000","last_fullsized":"2023-01-25T06:32:17.927167+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:35.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.756849+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.756514+0000","last_became_peered":"2023-01-25T06:32:16.756514+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:26.282967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927169+0000","last_change":"2023-01-25T06:32:15.673719+0000","last_active":"2023-01-25T06:32:17.927169+0000","last_peered":"2023-01-25T06:32:17.927169+0000","last_clean":"2023-01-25T06:32:17.927169+0000","last_became_active":"2023-01-25T06:32:15.673394+0000","last_became_peered":"2023-01-25T06:32:15.673394+0000","last_unstale":"2023-01-25T06:32:17.927169+0000","last_undegraded":"2023-01-25T06:32:17.927169+0000","last_fullsized":"2023-01-25T06:32:17.927169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:14.655937+0000","last_clean_scrub_stamp":"2023-01-25T06:32:14.655937+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:22.671489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927082+0000","last_change":"2023-01-25T06:32:16.692900+0000","last_active":"2023-01-25T06:32:17.927082+0000","last_peered":"2023-01-25T06:32:17.927082+0000","last_clean":"2023-01-25T06:32:17.927082+0000","last_became_active":"2023-01-25T06:32:16.692609+0000","last_became_peered":"2023-01-25T06:32:16.692609+0000","last_unstale":"2023-01-25T06:32:17.927082+0000","last_undegraded":"2023-01-25T06:32:17.927082+0000","last_fullsized":"2023-01-25T06:32:17.927082+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:44.684692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.960305+0000","last_change":"2023-01-25T06:32:16.756988+0000","last_active":"2023-01-25T06:32:17.960305+0000","last_peered":"2023-01-25T06:32:17.960305+0000","last_clean":"2023-01-25T06:32:17.960305+0000","last_became_active":"2023-01-25T06:32:16.756599+0000","last_became_peered":"2023-01-25T06:32:16.756599+0000","last_unstale":"2023-01-25T06:32:17.960305+0000","last_undegraded":"2023-01-25T06:32:17.960305+0000","last_fullsized":"2023-01-25T06:32:17.960305+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:23.668972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.927081+0000","last_change":"2023-01-25T06:32:16.692157+0000","last_active":"2023-01-25T06:32:17.927081+0000","last_peered":"2023-01-25T06:32:17.927081+0000","last_clean":"2023-01-25T06:32:17.927081+0000","last_became_active":"2023-01-25T06:32:16.691857+0000","last_became_peered":"2023-01-25T06:32:16.691857+0000","last_unstale":"2023-01-25T06:32:17.927081+0000","last_undegraded":"2023-01-25T06:32:17.927081+0000","last_fullsized":"2023-01-25T06:32:17.927081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:43.658061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-25T06:32:17.937951+0000","last_change":"2023-01-25T06:32:16.756859+0000","last_active":"2023-01-25T06:32:17.937951+0000","last_peered":"2023-01-25T06:32:17.937951+0000","last_clean":"2023-01-25T06:32:17.937951+0000","last_became_active":"2023-01-25T06:32:16.756503+0000","last_became_peered":"2023-01-25T06:32:16.756503+0000","last_unstale":"2023-01-25T06:32:17.937951+0000","last_undegraded":"2023-01-25T06:32:17.937951+0000","last_fullsized":"2023-01-25T06:32:17.937951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T06:32:15.662030+0000","last_clean_scrub_stamp":"2023-01-25T06:32:15.662030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T06:33:42.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6112,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365728,"statfs":{"total":96636764160,"available":96630505472,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4190000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6369999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6556,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":94365284,"statfs":{"total":96636764160,"available":96630050816,"internally_reserved":0,"allocated":679936,"data_stored":575118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.9339999999999993}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.9860000000000007}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9619999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1059999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1470000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5144,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":94366696,"statfs":{"total":96636764160,"available":96631496704,"internally_reserved":0,"allocated":147456,"data_stored":46190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T06:32:22.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T06:32:22.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-01-25T06:32:23.012 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 17 2023-01-25T06:32:23.014 INFO:teuthology.orchestra.run.smithi062.stdout:fsid b10aa3db-f868-4fb2-8582-d4662bf923a5 2023-01-25T06:32:23.015 INFO:teuthology.orchestra.run.smithi062.stdout:created 2023-01-25T06:32:04.249805+0000 2023-01-25T06:32:23.015 INFO:teuthology.orchestra.run.smithi062.stdout:modified 2023-01-25T06:32:17.686657+0000 2023-01-25T06:32:23.015 INFO:teuthology.orchestra.run.smithi062.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-01-25T06:32:23.016 INFO:teuthology.orchestra.run.smithi062.stdout:crush_version 4 2023-01-25T06:32:23.016 INFO:teuthology.orchestra.run.smithi062.stdout:full_ratio 0.95 2023-01-25T06:32:23.016 INFO:teuthology.orchestra.run.smithi062.stdout:backfillfull_ratio 0.9 2023-01-25T06:32:23.016 INFO:teuthology.orchestra.run.smithi062.stdout:nearfull_ratio 0.85 2023-01-25T06:32:23.018 INFO:teuthology.orchestra.run.smithi062.stdout:require_min_compat_client luminous 2023-01-25T06:32:23.018 INFO:teuthology.orchestra.run.smithi062.stdout:min_compat_client jewel 2023-01-25T06:32:23.018 INFO:teuthology.orchestra.run.smithi062.stdout:require_osd_release reef 2023-01-25T06:32:23.018 INFO:teuthology.orchestra.run.smithi062.stdout:stretch_mode_enabled false 2023-01-25T06:32:23.018 INFO:teuthology.orchestra.run.smithi062.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2023-01-25T06:32:23.019 INFO:teuthology.orchestra.run.smithi062.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd 2023-01-25T06:32:23.019 INFO:teuthology.orchestra.run.smithi062.stdout:max_osd 8 2023-01-25T06:32:23.019 INFO:teuthology.orchestra.run.smithi062.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6800/106361 v1:172.21.15.62:6801/106361 exists,up 35ab537b-0b4c-4201-8d19-ae2f26bb6bca 2023-01-25T06:32:23.019 INFO:teuthology.orchestra.run.smithi062.stdout:osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6812/106363 v1:172.21.15.62:6813/106363 exists,up f5427087-c30c-477b-8d8c-0d52faa69b7b 2023-01-25T06:32:23.019 INFO:teuthology.orchestra.run.smithi062.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6805/106364 v1:172.21.15.62:6807/106364 exists,up 3918efca-e289-4f0d-966d-f98eb66e12a9 2023-01-25T06:32:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6804/106362 v1:172.21.15.62:6806/106362 exists,up 601c9ac3-dc46-4084-ba43-a74aacb3b21c 2023-01-25T06:32:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6800/106280 v1:172.21.15.110:6801/106280 exists,up 814abf3f-0ec5-4f18-a7cb-35a080e3a176 2023-01-25T06:32:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6802/106279 v1:172.21.15.110:6805/106279 exists,up 9bc191b3-128d-418d-90d2-136a77a61e07 2023-01-25T06:32:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6812/106324 v1:172.21.15.110:6813/106324 exists,up d5aa11ca-8ff2-4f20-8678-50a2d97f36f2 2023-01-25T06:32:23.020 INFO:teuthology.orchestra.run.smithi062.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6808/106281 v1:172.21.15.110:6809/106281 exists,up 1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9 2023-01-25T06:32:23.027 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6800/106361 v1:172.21.15.62:6801/106361 exists,up 35ab537b-0b4c-4201-8d19-ae2f26bb6bca', 'osd.1 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6812/106363 v1:172.21.15.62:6813/106363 exists,up f5427087-c30c-477b-8d8c-0d52faa69b7b', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6805/106364 v1:172.21.15.62:6807/106364 exists,up 3918efca-e289-4f0d-966d-f98eb66e12a9', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.62:6804/106362 v1:172.21.15.62:6806/106362 exists,up 601c9ac3-dc46-4084-ba43-a74aacb3b21c', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6800/106280 v1:172.21.15.110:6801/106280 exists,up 814abf3f-0ec5-4f18-a7cb-35a080e3a176', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6802/106279 v1:172.21.15.110:6805/106279 exists,up 9bc191b3-128d-418d-90d2-136a77a61e07', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6812/106324 v1:172.21.15.110:6813/106324 exists,up d5aa11ca-8ff2-4f20-8678-50a2d97f36f2', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v1:172.21.15.110:6808/106281 v1:172.21.15.110:6809/106281 exists,up 1be18a1c-e6df-4cbb-9bb5-e0c8f87464d9'] 2023-01-25T06:32:23.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-01-25T06:32:23.153 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2023-01-25T06:32:23.154 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "mon.a", 2023-01-25T06:32:23.154 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster": "ceph", 2023-01-25T06:32:23.154 INFO:teuthology.orchestra.run.smithi062.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-01-25T06:32:23.154 INFO:teuthology.orchestra.run.smithi062.stdout: "admin_socket_mode": "", 2023-01-25T06:32:23.154 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-01-25T06:32:23.155 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_client_required": "cephx, none", 2023-01-25T06:32:23.155 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_cluster_required": "cephx", 2023-01-25T06:32:23.155 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_debug": "true", 2023-01-25T06:32:23.155 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-01-25T06:32:23.156 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-01-25T06:32:23.156 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_service_required": "cephx", 2023-01-25T06:32:23.156 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_service_ticket_ttl": "240.000000", 2023-01-25T06:32:23.156 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_supported": "cephx", 2023-01-25T06:32:23.156 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio": "true", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_max_queue_depth": "1024", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_poll_ms": "250", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_aio_reap_max": "16", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_async_discard": "false", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_block_size": "4096", 2023-01-25T06:32:23.157 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio": "false", 2023-01-25T06:32:23.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-01-25T06:32:23.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-01-25T06:32:23.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_debug_inflight_ios": "false", 2023-01-25T06:32:23.158 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_enable_discard": "false", 2023-01-25T06:32:23.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_flock_retry": "3", 2023-01-25T06:32:23.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_flock_retry_interval": "0.100000", 2023-01-25T06:32:23.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_inject_crash": "0", 2023-01-25T06:32:23.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_inject_crash_flush_delay": "2", 2023-01-25T06:32:23.159 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring": "false", 2023-01-25T06:32:23.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring_hipri": "false", 2023-01-25T06:32:23.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_ioring_sqthread_poll": "false", 2023-01-25T06:32:23.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-01-25T06:32:23.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_read_buffer_alignment": "4096", 2023-01-25T06:32:23.160 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-01-25T06:32:23.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bdev_type": "", 2023-01-25T06:32:23.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_alloc_size": "1048576", 2023-01-25T06:32:23.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_allocator": "hybrid", 2023-01-25T06:32:23.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_buffered_io": "true", 2023-01-25T06:32:23.161 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_for_zeros": "false", 2023-01-25T06:32:23.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_volume_selector_often": "false", 2023-01-25T06:32:23.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-01-25T06:32:23.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_compact_log_sync": "false", 2023-01-25T06:32:23.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-01-25T06:32:23.162 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-01-25T06:32:23.163 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_compact_min_size": "16777216", 2023-01-25T06:32:23.163 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_log_replay_check_allocations": "true", 2023-01-25T06:32:23.163 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_max_log_runway": "4194304", 2023-01-25T06:32:23.163 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_max_prefetch": "1048576", 2023-01-25T06:32:23.163 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_min_flush_size": "524288", 2023-01-25T06:32:23.164 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_min_log_runway": "1048576", 2023-01-25T06:32:23.164 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_replay_recovery": "false", 2023-01-25T06:32:23.164 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-01-25T06:32:23.165 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_shared_alloc_size": "65536", 2023-01-25T06:32:23.165 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_sync_write": "false", 2023-01-25T06:32:23.165 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-01-25T06:32:23.165 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-01-25T06:32:23.165 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-01-25T06:32:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_allocation_from_file": "true", 2023-01-25T06:32:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_allocator": "hybrid", 2023-01-25T06:32:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-01-25T06:32:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-01-25T06:32:23.166 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-01-25T06:32:23.167 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-01-25T06:32:23.167 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-01-25T06:32:23.167 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-01-25T06:32:23.167 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blobid_prealloc": "10240", 2023-01-25T06:32:23.167 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_create": "true", 2023-01-25T06:32:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_create": "false", 2023-01-25T06:32:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_path": "", 2023-01-25T06:32:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_db_size": "0", 2023-01-25T06:32:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_path": "", 2023-01-25T06:32:23.168 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_preallocate_file": "false", 2023-01-25T06:32:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_size": "107374182400", 2023-01-25T06:32:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_create": "false", 2023-01-25T06:32:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_path": "", 2023-01-25T06:32:23.169 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_block_wal_size": "100663296", 2023-01-25T06:32:23.170 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs": "true", 2023-01-25T06:32:23.170 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-01-25T06:32:23.170 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_env_mirror": "false", 2023-01-25T06:32:23.170 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-01-25T06:32:23.170 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-01-25T06:32:23.171 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T06:32:23.171 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T06:32:23.171 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-01-25T06:32:23.171 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-01-25T06:32:23.171 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_autotune": "true", 2023-01-25T06:32:23.172 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-01-25T06:32:23.172 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-01-25T06:32:23.172 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-01-25T06:32:23.172 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-01-25T06:32:23.172 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size": "0", 2023-01-25T06:32:23.173 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-01-25T06:32:23.173 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-01-25T06:32:23.173 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-01-25T06:32:23.173 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-01-25T06:32:23.173 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_type": "2q", 2023-01-25T06:32:23.174 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-01-25T06:32:23.174 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_clone_cow": "true", 2023-01-25T06:32:23.174 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_algorithm": "snappy", 2023-01-25T06:32:23.174 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size": "0", 2023-01-25T06:32:23.175 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-01-25T06:32:23.175 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-01-25T06:32:23.175 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size": "0", 2023-01-25T06:32:23.175 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-01-25T06:32:23.175 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-01-25T06:32:23.176 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_mode": "none", 2023-01-25T06:32:23.176 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-01-25T06:32:23.176 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_csum_type": "crc32c", 2023-01-25T06:32:23.176 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_enforce_settings": "default", 2023-01-25T06:32:23.176 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_freelist": "false", 2023-01-25T06:32:23.177 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_fsck_abort": "false", 2023-01-25T06:32:23.177 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-01-25T06:32:23.177 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_bug21040": "false", 2023-01-25T06:32:23.177 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-01-25T06:32:23.177 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_inject_read_err": "false", 2023-01-25T06:32:23.178 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_legacy_omap": "false", 2023-01-25T06:32:23.178 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-01-25T06:32:23.178 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-01-25T06:32:23.178 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-01-25T06:32:23.179 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-01-25T06:32:23.179 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_prefill": "0.000000", 2023-01-25T06:32:23.179 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-01-25T06:32:23.179 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-01-25T06:32:23.180 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-01-25T06:32:23.180 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_small_allocations": "0", 2023-01-25T06:32:23.180 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-01-25T06:32:23.180 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_default_buffered_read": "true", 2023-01-25T06:32:23.180 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_default_buffered_write": "false", 2023-01-25T06:32:23.181 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops": "0", 2023-01-25T06:32:23.181 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-01-25T06:32:23.181 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-01-25T06:32:23.181 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-01-25T06:32:23.181 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-01-25T06:32:23.182 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-01-25T06:32:23.182 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-01-25T06:32:23.182 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-01-25T06:32:23.182 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fail_eio": "false", 2023-01-25T06:32:23.182 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-01-25T06:32:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-01-25T06:32:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-01-25T06:32:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-01-25T06:32:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mkfs": "true", 2023-01-25T06:32:23.184 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-01-25T06:32:23.184 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mount": "false", 2023-01-25T06:32:23.184 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-01-25T06:32:23.184 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_umount": "false", 2023-01-25T06:32:23.184 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-01-25T06:32:23.185 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-01-25T06:32:23.185 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-01-25T06:32:23.185 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-01-25T06:32:23.185 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-01-25T06:32:23.185 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_ignore_data_csum": "false", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_kvbackend": "rocksdb", 2023-01-25T06:32:23.186 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-01-25T06:32:23.187 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-01-25T06:32:23.187 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_log_op_age": "5.000000", 2023-01-25T06:32:23.187 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_alloc_size": "0", 2023-01-25T06:32:23.187 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size": "0", 2023-01-25T06:32:23.187 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-01-25T06:32:23.188 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-01-25T06:32:23.188 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_defer_interval": "3.000000", 2023-01-25T06:32:23.188 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_max_deferred_txc": "32", 2023-01-25T06:32:23.188 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size": "0", 2023-01-25T06:32:23.188 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-01-25T06:32:23.189 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-01-25T06:32:23.189 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_nid_prealloc": "1024", 2023-01-25T06:32:23.189 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size": "0", 2023-01-25T06:32:23.189 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-01-25T06:32:23.189 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-01-25T06:32:23.190 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_qfsck_on_mount": "true", 2023-01-25T06:32:23.190 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_retry_disk_reads": "3", 2023-01-25T06:32:23.190 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_cf": "true", 2023-01-25T06:32:23.190 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2023-01-25T06:32:23.190 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=128,min_write_buffer_number_to_merge=16,compaction_style=kCompactionStyleLevel,write_buffer_size=8388608,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0,ttl=21600", 2023-01-25T06:32:23.191 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_rocksdb_options_annex": "", 2023-01-25T06:32:23.191 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_coremask": "0x1", 2023-01-25T06:32:23.191 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_io_sleep": "5", 2023-01-25T06:32:23.191 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_max_io_completion": "0", 2023-01-25T06:32:23.192 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_spdk_mem": "512", 2023-01-25T06:32:23.192 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_sync_submit_transaction": "false", 2023-01-25T06:32:23.192 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_bytes": "67108864", 2023-01-25T06:32:23.192 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io": "0", 2023-01-25T06:32:23.192 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-01-25T06:32:23.193 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-01-25T06:32:23.193 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-01-25T06:32:23.193 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-01-25T06:32:23.193 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_tracing": "false", 2023-01-25T06:32:23.193 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-01-25T06:32:23.194 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-01-25T06:32:23.194 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_reserved": "0", 2023-01-25T06:32:23.194 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-01-25T06:32:23.194 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-01-25T06:32:23.194 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-01-25T06:32:23.195 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-01-25T06:32:23.195 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-01-25T06:32:23.195 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-01-25T06:32:23.195 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_zero_block_detection": "false", 2023-01-25T06:32:23.196 INFO:teuthology.orchestra.run.smithi062.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-01-25T06:32:23.196 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_action_update_interval": "2", 2023-01-25T06:32:23.196 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-01-25T06:32:23.196 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-01-25T06:32:23.196 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-01-25T06:32:23.197 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-01-25T06:32:23.197 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_mount_timeout": "10", 2023-01-25T06:32:23.197 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-01-25T06:32:23.197 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-01-25T06:32:23.197 INFO:teuthology.orchestra.run.smithi062.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-01-25T06:32:23.198 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-01-25T06:32:23.198 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-01-25T06:32:23.198 INFO:teuthology.orchestra.run.smithi062.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-01-25T06:32:23.198 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_cluster_require_signatures": "false", 2023-01-25T06:32:23.198 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_cluster_require_version": "2", 2023-01-25T06:32:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_require_signatures": "false", 2023-01-25T06:32:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_require_version": "2", 2023-01-25T06:32:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_service_require_signatures": "false", 2023-01-25T06:32:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_service_require_version": "2", 2023-01-25T06:32:23.199 INFO:teuthology.orchestra.run.smithi062.stdout: "cephx_sign_messages": "true", 2023-01-25T06:32:23.200 INFO:teuthology.orchestra.run.smithi062.stdout: "chdir": "", 2023-01-25T06:32:23.200 INFO:teuthology.orchestra.run.smithi062.stdout: "client_acl_type": "", 2023-01-25T06:32:23.200 INFO:teuthology.orchestra.run.smithi062.stdout: "client_asio_thread_count": "2", 2023-01-25T06:32:23.200 INFO:teuthology.orchestra.run.smithi062.stdout: "client_cache_mid": "0.750000", 2023-01-25T06:32:23.200 INFO:teuthology.orchestra.run.smithi062.stdout: "client_cache_size": "16384", 2023-01-25T06:32:23.201 INFO:teuthology.orchestra.run.smithi062.stdout: "client_caps_release_delay": "5", 2023-01-25T06:32:23.201 INFO:teuthology.orchestra.run.smithi062.stdout: "client_check_pool_perm": "true", 2023-01-25T06:32:23.201 INFO:teuthology.orchestra.run.smithi062.stdout: "client_collect_and_send_global_metrics": "false", 2023-01-25T06:32:23.201 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_force_sync_read": "false", 2023-01-25T06:32:23.201 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_getattr_caps": "false", 2023-01-25T06:32:23.202 INFO:teuthology.orchestra.run.smithi062.stdout: "client_debug_inject_tick_delay": "0", 2023-01-25T06:32:23.202 INFO:teuthology.orchestra.run.smithi062.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-01-25T06:32:23.202 INFO:teuthology.orchestra.run.smithi062.stdout: "client_die_on_failed_remount": "false", 2023-01-25T06:32:23.202 INFO:teuthology.orchestra.run.smithi062.stdout: "client_dirsize_rbytes": "true", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_force_lazyio": "false", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_fs": "", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_inject_fixed_oldest_tid": "false", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_inject_release_failure": "false", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_max_inline_size": "4096", 2023-01-25T06:32:23.203 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mds_namespace": "", 2023-01-25T06:32:23.204 INFO:teuthology.orchestra.run.smithi062.stdout: "client_metadata": "", 2023-01-25T06:32:23.204 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_gid": "-1", 2023-01-25T06:32:23.204 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_timeout": "300", 2023-01-25T06:32:23.204 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mount_uid": "-1", 2023-01-25T06:32:23.205 INFO:teuthology.orchestra.run.smithi062.stdout: "client_mountpoint": "/", 2023-01-25T06:32:23.205 INFO:teuthology.orchestra.run.smithi062.stdout: "client_notify_timeout": "10", 2023-01-25T06:32:23.205 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc": "true", 2023-01-25T06:32:23.205 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_dirty": "104857600", 2023-01-25T06:32:23.205 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_dirty_age": "5.000000", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_max_objects": "1000", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_size": "209715200", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_oc_target_dirty": "8388608", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_permissions": "true", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_quota": "true", 2023-01-25T06:32:23.206 INFO:teuthology.orchestra.run.smithi062.stdout: "client_quota_df": "true", 2023-01-25T06:32:23.207 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_max_bytes": "0", 2023-01-25T06:32:23.207 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_max_periods": "4", 2023-01-25T06:32:23.207 INFO:teuthology.orchestra.run.smithi062.stdout: "client_readahead_min": "131072", 2023-01-25T06:32:23.207 INFO:teuthology.orchestra.run.smithi062.stdout: "client_reconnect_stale": "false", 2023-01-25T06:32:23.207 INFO:teuthology.orchestra.run.smithi062.stdout: "client_shutdown_timeout": "30", 2023-01-25T06:32:23.208 INFO:teuthology.orchestra.run.smithi062.stdout: "client_snapdir": ".snap", 2023-01-25T06:32:23.208 INFO:teuthology.orchestra.run.smithi062.stdout: "client_tick_interval": "1", 2023-01-25T06:32:23.208 INFO:teuthology.orchestra.run.smithi062.stdout: "client_trace": "", 2023-01-25T06:32:23.208 INFO:teuthology.orchestra.run.smithi062.stdout: "client_try_dentry_invalidate": "false", 2023-01-25T06:32:23.208 INFO:teuthology.orchestra.run.smithi062.stdout: "client_use_faked_inos": "false", 2023-01-25T06:32:23.209 INFO:teuthology.orchestra.run.smithi062.stdout: "client_use_random_mds": "false", 2023-01-25T06:32:23.209 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog": "false", 2023-01-25T06:32:23.209 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-01-25T06:32:23.209 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_graylog_port": "12201", 2023-01-25T06:32:23.209 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_monitors": "default=true", 2023-01-25T06:32:23.210 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog": "false", 2023-01-25T06:32:23.210 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-01-25T06:32:23.210 INFO:teuthology.orchestra.run.smithi062.stdout: "clog_to_syslog_level": "info", 2023-01-25T06:32:23.210 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_addr": "-", 2023-01-25T06:32:23.210 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_network": "", 2023-01-25T06:32:23.211 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_network_interface": "", 2023-01-25T06:32:23.211 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_isal": "false", 2023-01-25T06:32:23.211 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_level": "5", 2023-01-25T06:32:23.211 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zlib_winsize": "-15", 2023-01-25T06:32:23.211 INFO:teuthology.orchestra.run.smithi062.stdout: "compressor_zstd_level": "1", 2023-01-25T06:32:23.212 INFO:teuthology.orchestra.run.smithi062.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-01-25T06:32:23.212 INFO:teuthology.orchestra.run.smithi062.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-01-25T06:32:23.212 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_alien_op_num_threads": "6", 2023-01-25T06:32:23.212 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_alien_thread_cpu_cores": "", 2023-01-25T06:32:23.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_osd_obc_lru_size": "10", 2023-01-25T06:32:23.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-01-25T06:32:23.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crimson_seastar_smp": "1", 2023-01-25T06:32:23.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location": "", 2023-01-25T06:32:23.213 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location_hook": "", 2023-01-25T06:32:23.214 INFO:teuthology.orchestra.run.smithi062.stdout: "crush_location_hook_timeout": "10", 2023-01-25T06:32:23.214 INFO:teuthology.orchestra.run.smithi062.stdout: "daemonize": "false", 2023-01-25T06:32:23.214 INFO:teuthology.orchestra.run.smithi062.stdout: "daos_pool": "tank", 2023-01-25T06:32:23.214 INFO:teuthology.orchestra.run.smithi062.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-01-25T06:32:23.214 INFO:teuthology.orchestra.run.smithi062.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "dbstore_db_name_prefix": "dbstore", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_alienstore": "0/5", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_allow_any_pool_priority": "false", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asok": "1/5", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asok_assert_abort": "false", 2023-01-25T06:32:23.215 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_asserts_on_shutdown": "true", 2023-01-25T06:32:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_auth": "20/20", 2023-01-25T06:32:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bdev": "1/3", 2023-01-25T06:32:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bluefs": "1/5", 2023-01-25T06:32:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_bluestore": "1/5", 2023-01-25T06:32:23.216 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_buffer": "0/1", 2023-01-25T06:32:23.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_ceph_exporter": "1/5", 2023-01-25T06:32:23.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_cephfs_mirror": "0/5", 2023-01-25T06:32:23.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_cephsqlite": "0/5", 2023-01-25T06:32:23.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_client": "0/5", 2023-01-25T06:32:23.217 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_compressor": "1/5", 2023-01-25T06:32:23.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_context": "0/1", 2023-01-25T06:32:23.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_crush": "1/1", 2023-01-25T06:32:23.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_crypto": "1/5", 2023-01-25T06:32:23.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_cyanstore": "0/5", 2023-01-25T06:32:23.218 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_deliberately_leak_memory": "false", 2023-01-25T06:32:23.219 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_disable_randomized_ping": "false", 2023-01-25T06:32:23.219 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_dpdk": "1/5", 2023-01-25T06:32:23.219 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_eventtrace": "1/5", 2023-01-25T06:32:23.219 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_filer": "0/1", 2023-01-25T06:32:23.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_filestore": "1/3", 2023-01-25T06:32:23.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_finisher": "1/1", 2023-01-25T06:32:23.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_fuse": "1/5", 2023-01-25T06:32:23.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_heartbeat_testing_span": "0", 2023-01-25T06:32:23.220 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_heartbeatmap": "1/5", 2023-01-25T06:32:23.221 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_immutable_obj_cache": "0/5", 2023-01-25T06:32:23.221 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_javaclient": "1/5", 2023-01-25T06:32:23.221 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_journal": "1/3", 2023-01-25T06:32:23.221 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_journaler": "0/5", 2023-01-25T06:32:23.221 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_kstore": "1/5", 2023-01-25T06:32:23.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_leveldb": "4/5", 2023-01-25T06:32:23.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_lockdep": "0/1", 2023-01-25T06:32:23.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mclock": "1/5", 2023-01-25T06:32:23.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds": "1/5", 2023-01-25T06:32:23.222 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_balancer": "1/5", 2023-01-25T06:32:23.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_locker": "1/5", 2023-01-25T06:32:23.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_log": "1/5", 2023-01-25T06:32:23.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_log_expire": "1/5", 2023-01-25T06:32:23.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mds_migrator": "1/5", 2023-01-25T06:32:23.223 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_memstore": "1/5", 2023-01-25T06:32:23.224 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mgr": "2/5", 2023-01-25T06:32:23.224 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mgrc": "1/5", 2023-01-25T06:32:23.224 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_mon": "20/20", 2023-01-25T06:32:23.224 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_monc": "0/10", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_ms": "1/1", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_none": "0/5", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objclass": "0/5", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objectcacher": "0/5", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_objecter": "0/1", 2023-01-25T06:32:23.225 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_optracker": "0/5", 2023-01-25T06:32:23.226 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_osd": "1/5", 2023-01-25T06:32:23.226 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_paxos": "20/20", 2023-01-25T06:32:23.226 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_perfcounter": "1/5", 2023-01-25T06:32:23.226 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_prioritycache": "1/5", 2023-01-25T06:32:23.226 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rados": "0/5", 2023-01-25T06:32:23.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd": "0/5", 2023-01-25T06:32:23.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_mirror": "0/5", 2023-01-25T06:32:23.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_pwl": "0/5", 2023-01-25T06:32:23.227 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rbd_replay": "0/5", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_refs": "0/0", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_reserver": "1/1", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw": "1/5", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_access": "1/5", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_datacache": "1/5", 2023-01-25T06:32:23.228 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_dbstore": "1/5", 2023-01-25T06:32:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_flight": "1/5", 2023-01-25T06:32:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rgw_sync": "1/5", 2023-01-25T06:32:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_rocksdb": "4/5", 2023-01-25T06:32:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore": "0/5", 2023-01-25T06:32:23.229 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_backref": "0/5", 2023-01-25T06:32:23.230 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_cache": "0/5", 2023-01-25T06:32:23.230 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_cleaner": "0/5", 2023-01-25T06:32:23.230 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_device": "0/5", 2023-01-25T06:32:23.230 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_epm": "0/5", 2023-01-25T06:32:23.230 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-01-25T06:32:23.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_journal": "0/5", 2023-01-25T06:32:23.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_lba": "0/5", 2023-01-25T06:32:23.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_odata": "0/5", 2023-01-25T06:32:23.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_omap": "0/5", 2023-01-25T06:32:23.231 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_onode": "0/5", 2023-01-25T06:32:23.232 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_t": "0/5", 2023-01-25T06:32:23.232 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_seastore_tm": "0/5", 2023-01-25T06:32:23.232 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_striper": "0/1", 2023-01-25T06:32:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_test": "0/5", 2023-01-25T06:32:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_throttle": "1/1", 2023-01-25T06:32:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_timer": "0/1", 2023-01-25T06:32:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: "debug_tp": "0/5", 2023-01-25T06:32:23.233 INFO:teuthology.orchestra.run.smithi062.stdout: "device_failure_prediction_mode": "none", 2023-01-25T06:32:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-01-25T06:32:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-01-25T06:32:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_graylog": "false", 2023-01-25T06:32:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_journald": "false", 2023-01-25T06:32:23.234 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_stderr": "true", 2023-01-25T06:32:23.235 INFO:teuthology.orchestra.run.smithi062.stdout: "err_to_syslog": "false", 2023-01-25T06:32:23.235 INFO:teuthology.orchestra.run.smithi062.stdout: "event_tracing": "false", 2023-01-25T06:32:23.235 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_addr": "0.0.0.0", 2023-01-25T06:32:23.235 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_http_port": "9926", 2023-01-25T06:32:23.236 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_prio_limit": "5", 2023-01-25T06:32:23.236 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-01-25T06:32:23.236 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_sort_metrics": "true", 2023-01-25T06:32:23.236 INFO:teuthology.orchestra.run.smithi062.stdout: "exporter_stats_period": "5", 2023-01-25T06:32:23.236 INFO:teuthology.orchestra.run.smithi062.stdout: "fake_statfs_for_testing": "0", 2023-01-25T06:32:23.237 INFO:teuthology.orchestra.run.smithi062.stdout: "fatal_signal_handlers": "true", 2023-01-25T06:32:23.237 INFO:teuthology.orchestra.run.smithi062.stdout: "filer_max_purge_ops": "10", 2023-01-25T06:32:23.237 INFO:teuthology.orchestra.run.smithi062.stdout: "filer_max_truncate_ops": "128", 2023-01-25T06:32:23.237 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_apply_finisher_threads": "1", 2023-01-25T06:32:23.237 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_blackhole": "false", 2023-01-25T06:32:23.238 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_btrfs_clone_range": "true", 2023-01-25T06:32:23.238 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_btrfs_snap": "true", 2023-01-25T06:32:23.238 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_caller_concurrency": "10", 2023-01-25T06:32:23.238 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_collect_device_partition_information": "true", 2023-01-25T06:32:23.238 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_commit_timeout": "600.000000", 2023-01-25T06:32:23.239 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_inject_read_err": "false", 2023-01-25T06:32:23.239 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_omap_check": "false", 2023-01-25T06:32:23.239 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_random_read_err": "0.000000", 2023-01-25T06:32:23.239 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_debug_verify_split": "false", 2023-01-25T06:32:23.239 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_dump_file": "", 2023-01-25T06:32:23.240 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-01-25T06:32:23.240 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-01-25T06:32:23.240 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fadvise": "true", 2023-01-25T06:32:23.240 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fail_eio": "true", 2023-01-25T06:32:23.240 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fd_cache_shards": "16", 2023-01-25T06:32:23.241 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fd_cache_size": "128", 2023-01-25T06:32:23.241 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fiemap": "false", 2023-01-25T06:32:23.241 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fiemap_threshold": "4096", 2023-01-25T06:32:23.241 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-01-25T06:32:23.241 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_index_retry_probability": "0.000000", 2023-01-25T06:32:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_inject_stall": "0", 2023-01-25T06:32:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_parallel": "false", 2023-01-25T06:32:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_trailing": "false", 2023-01-25T06:32:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_journal_writeahead": "false", 2023-01-25T06:32:23.242 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_kill_at": "0", 2023-01-25T06:32:23.243 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-01-25T06:32:23.243 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size": "0", 2023-01-25T06:32:23.243 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-01-25T06:32:23.243 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-01-25T06:32:23.243 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs": "0", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_other": "2", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_sync_interval": "5.000000", 2023-01-25T06:32:23.244 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size": "0", 2023-01-25T06:32:23.245 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-01-25T06:32:23.245 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-01-25T06:32:23.245 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-01-25T06:32:23.245 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_merge_threshold": "-10", 2023-01-25T06:32:23.245 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_min_sync_interval": "0.010000", 2023-01-25T06:32:23.246 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_odsync_write": "false", 2023-01-25T06:32:23.246 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_backend": "rocksdb", 2023-01-25T06:32:23.246 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_backend_path": "", 2023-01-25T06:32:23.246 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_omap_header_cache_size": "1024", 2023-01-25T06:32:23.246 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_ondisk_finisher_threads": "1", 2023-01-25T06:32:23.247 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-01-25T06:32:23.247 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_thread_timeout": "60", 2023-01-25T06:32:23.247 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_op_threads": "2", 2023-01-25T06:32:23.247 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_punch_hole": "false", 2023-01-25T06:32:23.247 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-01-25T06:32:23.248 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-01-25T06:32:23.248 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-01-25T06:32:23.248 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-01-25T06:32:23.248 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-01-25T06:32:23.248 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_bytes": "104857600", 2023-01-25T06:32:23.249 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-01-25T06:32:23.249 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-01-25T06:32:23.249 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-01-25T06:32:23.249 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_queue_max_ops": "50", 2023-01-25T06:32:23.249 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-01-25T06:32:23.250 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_seek_data_hole": "false", 2023-01-25T06:32:23.250 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_sloppy_crc": "false", 2023-01-25T06:32:23.250 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-01-25T06:32:23.250 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_splice": "false", 2023-01-25T06:32:23.250 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_split_multiple": "2", 2023-01-25T06:32:23.251 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_split_rand_factor": "20", 2023-01-25T06:32:23.251 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_update_to": "1000", 2023-01-25T06:32:23.251 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-01-25T06:32:23.251 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-01-25T06:32:23.251 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-01-25T06:32:23.252 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-01-25T06:32:23.252 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-01-25T06:32:23.252 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-01-25T06:32:23.252 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_enable": "true", 2023-01-25T06:32:23.253 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-01-25T06:32:23.253 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-01-25T06:32:23.253 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-01-25T06:32:23.253 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-01-25T06:32:23.253 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-01-25T06:32:23.254 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-01-25T06:32:23.254 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_xfs_extsize": "false", 2023-01-25T06:32:23.254 INFO:teuthology.orchestra.run.smithi062.stdout: "filestore_zfs_snap": "false", 2023-01-25T06:32:23.254 INFO:teuthology.orchestra.run.smithi062.stdout: "fio_dir": "/tmp/fio", 2023-01-25T06:32:23.254 INFO:teuthology.orchestra.run.smithi062.stdout: "fsid": "b10aa3db-f868-4fb2-8582-d4662bf923a5", 2023-01-25T06:32:23.255 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_allow_other": "true", 2023-01-25T06:32:23.255 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_atomic_o_trunc": "true", 2023-01-25T06:32:23.255 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_big_writes": "true", 2023-01-25T06:32:23.255 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_debug": "false", 2023-01-25T06:32:23.255 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_default_permissions": "false", 2023-01-25T06:32:23.256 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_disable_pagecache": "false", 2023-01-25T06:32:23.256 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_max_write": "0", 2023-01-25T06:32:23.256 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_multithreaded": "true", 2023-01-25T06:32:23.256 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_require_active_mds": "true", 2023-01-25T06:32:23.256 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_set_user_groups": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_move": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_read": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_splice_write": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_syncfs_on_mksnap": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "fuse_use_invalidate_cb": "true", 2023-01-25T06:32:23.257 INFO:teuthology.orchestra.run.smithi062.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-01-25T06:32:23.258 INFO:teuthology.orchestra.run.smithi062.stdout: "gss_target_name": "ceph", 2023-01-25T06:32:23.258 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_file": "", 2023-01-25T06:32:23.258 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_inject_failure": "0", 2023-01-25T06:32:23.258 INFO:teuthology.orchestra.run.smithi062.stdout: "heartbeat_interval": "5", 2023-01-25T06:32:23.258 INFO:teuthology.orchestra.run.smithi062.stdout: "host": "smithi062", 2023-01-25T06:32:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-01-25T06:32:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-01-25T06:32:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_max_size": "1073741824", 2023-01-25T06:32:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-01-25T06:32:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-01-25T06:32:23.260 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-01-25T06:32:23.260 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-01-25T06:32:23.260 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-01-25T06:32:23.260 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-01-25T06:32:23.260 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-01-25T06:32:23.261 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-01-25T06:32:23.261 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-01-25T06:32:23.261 INFO:teuthology.orchestra.run.smithi062.stdout: "immutable_object_cache_watermark": "0.900000", 2023-01-25T06:32:23.261 INFO:teuthology.orchestra.run.smithi062.stdout: "inject_early_sigterm": "false", 2023-01-25T06:32:23.261 INFO:teuthology.orchestra.run.smithi062.stdout: "jaeger_tracing_enable": "false", 2023-01-25T06:32:23.262 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_aio": "true", 2023-01-25T06:32:23.262 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_align_min_size": "65536", 2023-01-25T06:32:23.262 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_block_align": "true", 2023-01-25T06:32:23.262 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_block_size": "4096", 2023-01-25T06:32:23.262 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_dio": "true", 2023-01-25T06:32:23.263 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_discard": "false", 2023-01-25T06:32:23.263 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_force_aio": "false", 2023-01-25T06:32:23.263 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_ignore_corruption": "false", 2023-01-25T06:32:23.263 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_max_write_bytes": "10485760", 2023-01-25T06:32:23.263 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_max_write_entries": "100", 2023-01-25T06:32:23.264 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_replay_from": "0", 2023-01-25T06:32:23.264 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_high_multiple": "0.000000", 2023-01-25T06:32:23.264 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-01-25T06:32:23.264 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-01-25T06:32:23.264 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_throttle_max_multiple": "0.000000", 2023-01-25T06:32:23.265 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_write_header_frequency": "0", 2023-01-25T06:32:23.265 INFO:teuthology.orchestra.run.smithi062.stdout: "journal_zero_on_create": "false", 2023-01-25T06:32:23.265 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_prefetch_periods": "10", 2023-01-25T06:32:23.265 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_prezero_periods": "5", 2023-01-25T06:32:23.265 INFO:teuthology.orchestra.run.smithi062.stdout: "journaler_write_head_interval": "15", 2023-01-25T06:32:23.266 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "", 2023-01-25T06:32:23.266 INFO:teuthology.orchestra.run.smithi062.stdout: "keyfile": "", 2023-01-25T06:32:23.266 INFO:teuthology.orchestra.run.smithi062.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-01-25T06:32:23.266 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_backend": "rocksdb", 2023-01-25T06:32:23.266 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_default_stripe_size": "65536", 2023-01-25T06:32:23.267 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_fsck_on_mount": "false", 2023-01-25T06:32:23.267 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_fsck_on_mount_deep": "true", 2023-01-25T06:32:23.267 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_max_bytes": "67108864", 2023-01-25T06:32:23.267 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_max_ops": "512", 2023-01-25T06:32:23.267 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_nid_prealloc": "1024", 2023-01-25T06:32:23.268 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_onode_map_size": "1024", 2023-01-25T06:32:23.268 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-01-25T06:32:23.268 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_sync_submit_transaction": "false", 2023-01-25T06:32:23.268 INFO:teuthology.orchestra.run.smithi062.stdout: "kstore_sync_transaction": "false", 2023-01-25T06:32:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_block_size": "65536", 2023-01-25T06:32:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_bloom_size": "0", 2023-01-25T06:32:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_cache_size": "536870912", 2023-01-25T06:32:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_compact_on_mount": "false", 2023-01-25T06:32:23.269 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_compression": "false", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_log": "", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_log_to_ceph_log": "true", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_max_open_files": "0", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_paranoid": "false", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "leveldb_write_buffer_size": "33554432", 2023-01-25T06:32:23.270 INFO:teuthology.orchestra.run.smithi062.stdout: "librados_thread_count": "2", 2023-01-25T06:32:23.271 INFO:teuthology.orchestra.run.smithi062.stdout: "lockdep": "false", 2023-01-25T06:32:23.271 INFO:teuthology.orchestra.run.smithi062.stdout: "lockdep_force_backtrace": "false", 2023-01-25T06:32:23.271 INFO:teuthology.orchestra.run.smithi062.stdout: "log_coarse_timestamps": "true", 2023-01-25T06:32:23.271 INFO:teuthology.orchestra.run.smithi062.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-01-25T06:32:23.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_flush_on_exit": "false", 2023-01-25T06:32:23.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_graylog_host": "127.0.0.1", 2023-01-25T06:32:23.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_graylog_port": "12201", 2023-01-25T06:32:23.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_max_new": "1000", 2023-01-25T06:32:23.272 INFO:teuthology.orchestra.run.smithi062.stdout: "log_max_recent": "10000", 2023-01-25T06:32:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_stderr_prefix": "", 2023-01-25T06:32:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_stop_at_utilization": "0.970000", 2023-01-25T06:32:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_file": "true", 2023-01-25T06:32:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_graylog": "false", 2023-01-25T06:32:23.273 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_journald": "false", 2023-01-25T06:32:23.274 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_stderr": "false", 2023-01-25T06:32:23.274 INFO:teuthology.orchestra.run.smithi062.stdout: "log_to_syslog": "false", 2023-01-25T06:32:23.274 INFO:teuthology.orchestra.run.smithi062.stdout: "max_rotating_auth_attempts": "10", 2023-01-25T06:32:23.274 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_action_on_write_error": "1", 2023-01-25T06:32:23.274 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_alternate_name_max": "8192", 2023-01-25T06:32:23.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_asio_thread_count": "2", 2023-01-25T06:32:23.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_export_pin": "true", 2023-01-25T06:32:23.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_dirs": "true", 2023-01-25T06:32:23.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-01-25T06:32:23.275 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_interval": "5", 2023-01-25T06:32:23.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_fragment_size_max": "100000", 2023-01-25T06:32:23.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_idle_threshold": "0.000000", 2023-01-25T06:32:23.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_interval": "10", 2023-01-25T06:32:23.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_max": "-1", 2023-01-25T06:32:23.276 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_max_until": "-1", 2023-01-25T06:32:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_merge_size": "50", 2023-01-25T06:32:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_midchunk": "0.300000", 2023-01-25T06:32:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_min_rebalance": "0.100000", 2023-01-25T06:32:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_min_start": "0.200000", 2023-01-25T06:32:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_minchunk": "0.001000", 2023-01-25T06:32:23.278 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_mode": "0", 2023-01-25T06:32:23.278 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_need_max": "1.200000", 2023-01-25T06:32:23.278 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_need_min": "0.800000", 2023-01-25T06:32:23.278 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-01-25T06:32:23.278 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_sample_interval": "3.000000", 2023-01-25T06:32:23.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_bits": "3", 2023-01-25T06:32:23.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_rd": "25000.000000", 2023-01-25T06:32:23.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_size": "10000", 2023-01-25T06:32:23.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_split_wr": "10000.000000", 2023-01-25T06:32:23.279 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_target_decay": "10.000000", 2023-01-25T06:32:23.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-01-25T06:32:23.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_grace": "15.000000", 2023-01-25T06:32:23.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_interval": "4.000000", 2023-01-25T06:32:23.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_beacon_mon_down_grace": "60", 2023-01-25T06:32:23.280 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_memory_limit": "4294967296", 2023-01-25T06:32:23.281 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_mid": "0.700000", 2023-01-25T06:32:23.281 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_release_free_interval": "10", 2023-01-25T06:32:23.281 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_reservation": "0.050000", 2023-01-25T06:32:23.281 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-01-25T06:32:23.281 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_interval": "1", 2023-01-25T06:32:23.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache_trim_threshold": "262144", 2023-01-25T06:32:23.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-01-25T06:32:23.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-01-25T06:32:23.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_delegate_inos_pct": "50", 2023-01-25T06:32:23.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_prealloc_inos": "1000", 2023-01-25T06:32:23.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-01-25T06:32:23.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_connect_bootstrapping": "false", 2023-01-25T06:32:23.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_damage_table_max_entries": "10000", 2023-01-25T06:32:23.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-01-25T06:32:23.283 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_auth_pins": "false", 2023-01-25T06:32:23.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_frag": "false", 2023-01-25T06:32:23.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_scatterstat": "false", 2023-01-25T06:32:23.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_debug_subtrees": "false", 2023-01-25T06:32:23.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_decay_halflife": "5.000000", 2023-01-25T06:32:23.284 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_default_dir_hash": "2", 2023-01-25T06:32:23.285 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_defer_session_stale": "true", 2023-01-25T06:32:23.285 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_deny_all_reconnect": "false", 2023-01-25T06:32:23.285 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_keys_per_op": "16384", 2023-01-25T06:32:23.285 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_max_commit_size": "10", 2023-01-25T06:32:23.285 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_max_entries": "0", 2023-01-25T06:32:23.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dir_prefetch": "true", 2023-01-25T06:32:23.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dirstat_min_interval": "1.000000", 2023-01-25T06:32:23.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_after_rejoin": "false", 2023-01-25T06:32:23.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_on_map": "false", 2023-01-25T06:32:23.286 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_threshold_file": "0", 2023-01-25T06:32:23.287 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-01-25T06:32:23.287 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_early_reply": "true", 2023-01-25T06:32:23.287 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_enable_op_tracker": "true", 2023-01-25T06:32:23.287 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_enforce_unique_name": "true", 2023-01-25T06:32:23.287 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_distributed": "true", 2023-01-25T06:32:23.288 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-01-25T06:32:23.288 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_random": "true", 2023-01-25T06:32:23.288 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-01-25T06:32:23.288 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-01-25T06:32:23.288 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_forward_all_requests_to_auth": "false", 2023-01-25T06:32:23.289 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-01-25T06:32:23.289 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-01-25T06:32:23.289 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-01-25T06:32:23.289 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_health_cache_threshold": "1.500000", 2023-01-25T06:32:23.289 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_health_summarize_threshold": "10", 2023-01-25T06:32:23.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_heartbeat_grace": "15.000000", 2023-01-25T06:32:23.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_heartbeat_reset_grace": "1000", 2023-01-25T06:32:23.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_health_dummy": "false", 2023-01-25T06:32:23.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_migrator_session_race": "false", 2023-01-25T06:32:23.290 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-01-25T06:32:23.291 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_join_fs": "", 2023-01-25T06:32:23.291 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_journal_format": "1", 2023-01-25T06:32:23.291 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_create_at": "0", 2023-01-25T06:32:23.291 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_export_at": "0", 2023-01-25T06:32:23.291 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_import_at": "0", 2023-01-25T06:32:23.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_at": "0", 2023-01-25T06:32:23.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_expire_at": "0", 2023-01-25T06:32:23.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_journal_replay_at": "0", 2023-01-25T06:32:23.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_link_at": "0", 2023-01-25T06:32:23.292 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_mdstable_at": "0", 2023-01-25T06:32:23.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_openc_at": "0", 2023-01-25T06:32:23.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_kill_rename_at": "0", 2023-01-25T06:32:23.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_events_per_segment": "1024", 2023-01-25T06:32:23.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_max_events": "-1", 2023-01-25T06:32:23.293 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_max_segments": "128", 2023-01-25T06:32:23.294 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_pause": "false", 2023-01-25T06:32:23.294 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_segment_size": "0", 2023-01-25T06:32:23.294 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_skip_corrupt_events": "false", 2023-01-25T06:32:23.294 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log_warn_factor": "2.000000", 2023-01-25T06:32:23.294 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_caps_per_client": "1048576", 2023-01-25T06:32:23.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_completed_flushes": "100000", 2023-01-25T06:32:23.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_completed_requests": "100000", 2023-01-25T06:32:23.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_export_size": "20971520", 2023-01-25T06:32:23.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_file_recover": "32", 2023-01-25T06:32:23.295 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_files": "64", 2023-01-25T06:32:23.296 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_ops": "8192", 2023-01-25T06:32:23.296 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-01-25T06:32:23.296 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_retries_on_remount_failure": "5", 2023-01-25T06:32:23.296 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-01-25T06:32:23.296 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_snaps_per_dir": "100", 2023-01-25T06:32:23.297 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_max_xattr_pairs_size": "65536", 2023-01-25T06:32:23.297 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_metrics_update_interval": "2", 2023-01-25T06:32:23.297 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_min_caps_per_client": "100", 2023-01-25T06:32:23.297 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_min_caps_working_set": "10000", 2023-01-25T06:32:23.298 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-01-25T06:32:23.298 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_numa_node": "-1", 2023-01-25T06:32:23.298 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-01-25T06:32:23.298 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_complaint_time": "30.000000", 2023-01-25T06:32:23.298 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_history_duration": "600", 2023-01-25T06:32:23.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_history_size": "20", 2023-01-25T06:32:23.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_op_log_threshold": "5", 2023-01-25T06:32:23.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_ping_grace": "15", 2023-01-25T06:32:23.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_ping_interval": "5", 2023-01-25T06:32:23.299 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-01-25T06:32:23.300 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-01-25T06:32:23.300 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_caps": "30000", 2023-01-25T06:32:23.300 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-01-25T06:32:23.300 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_max_decay_threshold": "131072", 2023-01-25T06:32:23.301 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-01-25T06:32:23.301 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_recall_warning_threshold": "262144", 2023-01-25T06:32:23.301 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_reconnect_timeout": "45.000000", 2023-01-25T06:32:23.301 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_replay_interval": "1.000000", 2023-01-25T06:32:23.301 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-01-25T06:32:23.302 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-01-25T06:32:23.302 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_root_ino_gid": "0", 2023-01-25T06:32:23.302 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_root_ino_uid": "0", 2023-01-25T06:32:23.302 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-01-25T06:32:23.303 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_blocklist_on_evict": "true", 2023-01-25T06:32:23.303 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_blocklist_on_timeout": "true", 2023-01-25T06:32:23.303 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-01-25T06:32:23.303 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-01-25T06:32:23.303 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cap_acquisition_decay_rate": "10.000000", 2023-01-25T06:32:23.304 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_cap_acquisition_throttle": "500000", 2023-01-25T06:32:23.304 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-01-25T06:32:23.304 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-01-25T06:32:23.304 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_shutdown_check": "0", 2023-01-25T06:32:23.304 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_skip_ino": "0", 2023-01-25T06:32:23.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sleep_rank_change": "0.000000", 2023-01-25T06:32:23.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_max_uid": "4294967294", 2023-01-25T06:32:23.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_min_uid": "0", 2023-01-25T06:32:23.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_snap_rstat": "false", 2023-01-25T06:32:23.305 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_standby_replay_damaged": "false", 2023-01-25T06:32:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_symlink_recovery": "true", 2023-01-25T06:32:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_task_status_update_interval": "2.000000", 2023-01-25T06:32:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_thrash_exports": "0", 2023-01-25T06:32:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_thrash_fragments": "0", 2023-01-25T06:32:23.306 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_tick_interval": "5.000000", 2023-01-25T06:32:23.307 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_valgrind_exit": "false", 2023-01-25T06:32:23.307 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_verify_backtrace": "1", 2023-01-25T06:32:23.307 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_verify_scatter": "false", 2023-01-25T06:32:23.307 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_wipe_ino_prealloc": "false", 2023-01-25T06:32:23.307 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_wipe_sessions": "false", 2023-01-25T06:32:23.308 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool_debug": "false", 2023-01-25T06:32:23.308 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_debug_omit_block_device_write": "false", 2023-01-25T06:32:23.308 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_device_bytes": "1073741824", 2023-01-25T06:32:23.308 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_page_set": "false", 2023-01-25T06:32:23.308 INFO:teuthology.orchestra.run.smithi062.stdout: "memstore_page_size": "65536", 2023-01-25T06:32:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_bytes": "134217728", 2023-01-25T06:32:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_messages": "512", 2023-01-25T06:32:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-01-25T06:32:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_connect_retry_interval": "1.000000", 2023-01-25T06:32:23.309 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-01-25T06:32:23.310 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-01-25T06:32:23.310 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_disabled_modules": "", 2023-01-25T06:32:23.310 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-01-25T06:32:23.310 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_max_pg_num_change": "128", 2023-01-25T06:32:23.310 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mds_bytes": "134217728", 2023-01-25T06:32:23.311 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mds_messages": "128", 2023-01-25T06:32:23.311 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-01-25T06:32:23.311 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mon_bytes": "134217728", 2023-01-25T06:32:23.311 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_mon_messages": "128", 2023-01-25T06:32:23.311 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_osd_bytes": "536870912", 2023-01-25T06:32:23.312 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_osd_messages": "8192", 2023-01-25T06:32:23.312 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_pool": "true", 2023-01-25T06:32:23.312 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_service_beacon_grace": "60.000000", 2023-01-25T06:32:23.312 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_standby_modules": "true", 2023-01-25T06:32:23.312 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_stats_period": "5", 2023-01-25T06:32:23.313 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_stats_threshold": "5", 2023-01-25T06:32:23.313 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_tick_period": "2", 2023-01-25T06:32:23.313 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-01-25T06:32:23.313 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_accept_timeout_factor": "2.000000", 2023-01-25T06:32:23.313 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_allow_pool_delete": "true", 2023-01-25T06:32:23.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_allow_pool_size_one": "true", 2023-01-25T06:32:23.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_auth_validate_all_caps": "true", 2023-01-25T06:32:23.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-01-25T06:32:23.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-01-25T06:32:23.314 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_bytes": "104857600", 2023-01-25T06:32:23.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_directed_command_retry": "5", 2023-01-25T06:32:23.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval": "3.000000", 2023-01-25T06:32:23.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-01-25T06:32:23.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-01-25T06:32:23.315 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-01-25T06:32:23.316 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_hunt_parallel": "3", 2023-01-25T06:32:23.316 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_log_interval": "1.000000", 2023-01-25T06:32:23.316 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-01-25T06:32:23.316 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_ping_interval": "10.000000", 2023-01-25T06:32:23.316 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_client_ping_timeout": "30.000000", 2023-01-25T06:32:23.317 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clock_drift_allowed": "1.000000", 2023-01-25T06:32:23.317 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-01-25T06:32:23.317 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-01-25T06:32:23.317 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_file_level": "debug", 2023-01-25T06:32:23.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_file": "true", 2023-01-25T06:32:23.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog": "false", 2023-01-25T06:32:23.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-01-25T06:32:23.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-01-25T06:32:23.318 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_journald": "false", 2023-01-25T06:32:23.319 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_stderr": "false", 2023-01-25T06:32:23.319 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-01-25T06:32:23.319 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-01-25T06:32:23.319 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-01-25T06:32:23.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_bootstrap": "false", 2023-01-25T06:32:23.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_start": "false", 2023-01-25T06:32:23.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_compact_on_trim": "true", 2023-01-25T06:32:23.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_con_tracker_persist_interval": "10", 2023-01-25T06:32:23.320 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_con_tracker_score_halflife": "43200", 2023-01-25T06:32:23.321 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_config_key_max_entry_size": "65536", 2023-01-25T06:32:23.321 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_cpu_threads": "4", 2023-01-25T06:32:23.321 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_crush_min_required_version": "hammer", 2023-01-25T06:32:23.321 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_daemon_bytes": "419430400", 2023-01-25T06:32:23.322 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-01-25T06:32:23.322 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_avail_crit": "5", 2023-01-25T06:32:23.322 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_avail_warn": "5", 2023-01-25T06:32:23.322 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_data_size_warn": "16106127360", 2023-01-25T06:32:23.323 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_block_osdmap_trim": "false", 2023-01-25T06:32:23.323 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-01-25T06:32:23.323 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_json": "false", 2023-01-25T06:32:23.323 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-01-25T06:32:23.324 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_dump_transactions": "false", 2023-01-25T06:32:23.324 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_extra_checks": "false", 2023-01-25T06:32:23.324 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-01-25T06:32:23.324 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-01-25T06:32:23.324 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_quincy": "false", 2023-01-25T06:32:23.325 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_no_require_reef": "false", 2023-01-25T06:32:23.325 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-01-25T06:32:23.325 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_delta_reset_interval": "10.000000", 2023-01-25T06:32:23.325 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_dns_srv_name": "ceph-mon", 2023-01-25T06:32:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_down_mkfs_grace": "120", 2023-01-25T06:32:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_election_default_strategy": "3", 2023-01-25T06:32:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_election_timeout": "5.000000", 2023-01-25T06:32:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-01-25T06:32:23.326 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ping_divisor": "2", 2023-01-25T06:32:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_elector_ping_timeout": "2.000000", 2023-01-25T06:32:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_enable_op_tracker": "true", 2023-01-25T06:32:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_fake_pool_delete": "false", 2023-01-25T06:32:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_force_quorum_join": "false", 2023-01-25T06:32:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_globalid_prealloc": "10000", 2023-01-25T06:32:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_detail_to_clog": "false", 2023-01-25T06:32:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_log_update_period": "5", 2023-01-25T06:32:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_max_detail": "50", 2023-01-25T06:32:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog": "true", 2023-01-25T06:32:23.328 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog_interval": "600", 2023-01-25T06:32:23.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-01-25T06:32:23.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_host": "[v1:172.21.15.62:6789],[v1:172.21.15.62:6790],[v1:172.21.15.110:6789]", 2023-01-25T06:32:23.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_host_override": "", 2023-01-25T06:32:23.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_initial_members": "", 2023-01-25T06:32:23.329 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-01-25T06:32:23.330 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-01-25T06:32:23.330 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-01-25T06:32:23.330 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-01-25T06:32:23.330 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_keyvaluedb": "rocksdb", 2023-01-25T06:32:23.330 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease": "5.000000", 2023-01-25T06:32:23.331 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-01-25T06:32:23.331 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-01-25T06:32:23.331 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_full_interval": "50", 2023-01-25T06:32:23.331 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_max": "10000", 2023-01-25T06:32:23.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_log_max_summary": "50", 2023-01-25T06:32:23.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_log_entries_per_event": "4096", 2023-01-25T06:32:23.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_log_epochs": "500", 2023-01-25T06:32:23.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_mdsmap_epochs": "500", 2023-01-25T06:32:23.332 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_mgrmap_epochs": "500", 2023-01-25T06:32:23.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_osd": "10000", 2023-01-25T06:32:23.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_pg_per_osd": "10000", 2023-01-25T06:32:23.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_pool_pg_num": "65536", 2023-01-25T06:32:23.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-01-25T06:32:23.333 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-01-25T06:32:23.334 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_force_trim_to": "0", 2023-01-25T06:32:23.334 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mds_skip_sanity": "false", 2023-01-25T06:32:23.334 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_memory_autotune": "true", 2023-01-25T06:32:23.334 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_memory_target": "2147483648", 2023-01-25T06:32:23.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_beacon_grace": "30", 2023-01-25T06:32:23.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-01-25T06:32:23.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_digest_period": "5", 2023-01-25T06:32:23.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_inactive_grace": "60", 2023-01-25T06:32:23.335 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_mkfs_grace": "240", 2023-01-25T06:32:23.336 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-01-25T06:32:23.336 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_min_osdmap_epochs": "500", 2023-01-25T06:32:23.336 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_complaint_time": "30", 2023-01-25T06:32:23.336 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_duration": "600", 2023-01-25T06:32:23.337 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_size": "20", 2023-01-25T06:32:23.337 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_slow_op_size": "20", 2023-01-25T06:32:23.337 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_history_slow_op_threshold": "10", 2023-01-25T06:32:23.337 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_op_log_threshold": "5", 2023-01-25T06:32:23.337 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-01-25T06:32:23.338 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-01-25T06:32:23.338 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-01-25T06:32:23.338 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_in": "false", 2023-01-25T06:32:23.338 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_auto_mark_new_in": "true", 2023-01-25T06:32:23.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-01-25T06:32:23.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-01-25T06:32:23.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_cache_size": "500", 2023-01-25T06:32:23.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_cache_size_min": "134217728", 2023-01-25T06:32:23.339 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_crush_smoke_test": "true", 2023-01-25T06:32:23.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_destroyed_out_interval": "600", 2023-01-25T06:32:23.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_down_out_interval": "600", 2023-01-25T06:32:23.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-01-25T06:32:23.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-01-25T06:32:23.340 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_force_trim_to": "0", 2023-01-25T06:32:23.341 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_full_ratio": "0.950000", 2023-01-25T06:32:23.341 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-01-25T06:32:23.341 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_halflife": "3600", 2023-01-25T06:32:23.341 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_max_interval": "300", 2023-01-25T06:32:23.341 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_laggy_weight": "0.300000", 2023-01-25T06:32:23.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-01-25T06:32:23.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_max_creating_pgs": "1024", 2023-01-25T06:32:23.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_max_initial_pgs": "1024", 2023-01-25T06:32:23.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_down_reporters": "2", 2023-01-25T06:32:23.342 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-01-25T06:32:23.343 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-01-25T06:32:23.343 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-01-25T06:32:23.343 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp": "true", 2023-01-25T06:32:23.343 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-01-25T06:32:23.343 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-01-25T06:32:23.344 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_report_timeout": "900", 2023-01-25T06:32:23.344 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-01-25T06:32:23.344 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-01-25T06:32:23.344 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_warn_num_repaired": "10", 2023-01-25T06:32:23.344 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osd_warn_op_age": "32.000000", 2023-01-25T06:32:23.345 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-01-25T06:32:23.345 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_interval": "10", 2023-01-25T06:32:23.345 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_min": "10000", 2023-01-25T06:32:23.345 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_osdmap_full_prune_txsize": "100", 2023-01-25T06:32:23.345 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-01-25T06:32:23.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_stuck_threshold": "60", 2023-01-25T06:32:23.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-01-25T06:32:23.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_objects": "10000", 2023-01-25T06:32:23.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_per_osd": "0", 2023-01-25T06:32:23.346 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-01-25T06:32:23.347 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pool_quota_crit_threshold": "0", 2023-01-25T06:32:23.347 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_pool_quota_warn_threshold": "0", 2023-01-25T06:32:23.347 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_probe_timeout": "2.000000", 2023-01-25T06:32:23.347 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_max_change": "0.050000", 2023-01-25T06:32:23.347 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_max_osds": "4", 2023-01-25T06:32:23.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-01-25T06:32:23.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-01-25T06:32:23.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-01-25T06:32:23.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-01-25T06:32:23.348 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-01-25T06:32:23.349 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_interval": "300", 2023-01-25T06:32:23.349 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_max_keys": "100", 2023-01-25T06:32:23.349 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_scrub_timeout": "300", 2023-01-25T06:32:23.349 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_session_timeout": "300", 2023-01-25T06:32:23.349 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_smart_report_timeout": "5", 2023-01-25T06:32:23.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stat_smooth_intervals": "6", 2023-01-25T06:32:23.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-01-25T06:32:23.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_pool_min_size": "2", 2023-01-25T06:32:23.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_pool_size": "4", 2023-01-25T06:32:23.350 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-01-25T06:32:23.351 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_subscribe_interval": "86400.000000", 2023-01-25T06:32:23.351 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_debug": "false", 2023-01-25T06:32:23.351 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_max_payload_keys": "2000", 2023-01-25T06:32:23.351 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_max_payload_size": "1048576", 2023-01-25T06:32:23.351 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_provider_kill_at": "0", 2023-01-25T06:32:23.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_requester_kill_at": "0", 2023-01-25T06:32:23.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_sync_timeout": "60.000000", 2023-01-25T06:32:23.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_target_pg_per_osd": "100", 2023-01-25T06:32:23.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_tick_interval": "5", 2023-01-25T06:32:23.352 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_timecheck_interval": "300.000000", 2023-01-25T06:32:23.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-01-25T06:32:23.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_use_min_delay_socket": "false", 2023-01-25T06:32:23.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_older_version_delay": "604800", 2023-01-25T06:32:23.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-01-25T06:32:23.353 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-01-25T06:32:23.354 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-01-25T06:32:23.354 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_filestore_osds": "false", 2023-01-25T06:32:23.354 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-01-25T06:32:23.354 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-01-25T06:32:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-01-25T06:32:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_misplaced": "false", 2023-01-25T06:32:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-01-25T06:32:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_older_version": "true", 2023-01-25T06:32:23.355 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-01-25T06:32:23.356 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_no_app": "true", 2023-01-25T06:32:23.356 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-01-25T06:32:23.356 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-01-25T06:32:23.356 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-01-25T06:32:23.356 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-01-25T06:32:23.357 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_on_too_few_osds": "false", 2023-01-25T06:32:23.357 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-01-25T06:32:23.357 INFO:teuthology.orchestra.run.smithi062.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-01-25T06:32:23.357 INFO:teuthology.orchestra.run.smithi062.stdout: "monmap": "", 2023-01-25T06:32:23.357 INFO:teuthology.orchestra.run.smithi062.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-01-25T06:32:23.358 INFO:teuthology.orchestra.run.smithi062.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-01-25T06:32:23.358 INFO:teuthology.orchestra.run.smithi062.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-01-25T06:32:23.358 INFO:teuthology.orchestra.run.smithi062.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-01-25T06:32:23.358 INFO:teuthology.orchestra.run.smithi062.stdout: "motr_tracing_enabled": "false", 2023-01-25T06:32:23.358 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_op_threads": "3", 2023-01-25T06:32:23.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_buffer_size": "131072", 2023-01-25T06:32:23.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_cm": "false", 2023-01-25T06:32:23.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_device_name": "", 2023-01-25T06:32:23.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_dscp": "96", 2023-01-25T06:32:23.359 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-01-25T06:32:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_gid_idx": "0", 2023-01-25T06:32:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_local_gid": "", 2023-01-25T06:32:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_polling_us": "1000", 2023-01-25T06:32:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_port_num": "1", 2023-01-25T06:32:23.360 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-01-25T06:32:23.361 INFO:teuthology.orchestra.run.smithi062.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-01-25T06:32:23.361 INFO:teuthology.orchest