2024-09-21T21:09:19.049 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T21:09:19.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T21:09:19.158 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914330 branch: main description: crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_read} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914330' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug bluefs: 20 debug bluestore: 20 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH - \(POOL_APP_NOT_ENABLED\) sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2950 sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 sleep_before_teardown: 0 subset: 0/1 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP4HkVd7XU6TAfrQnDFwZO2s5In9BRKqbdvgDi6RoPsjWkjQkQdhZRE0sNwlvEpKJYnqxZV8m8wNVZQVHDjSmAc= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson wait-for-scrub: false - cbt: benchmarks: librbdfio: iodepth: - 32 log_avg_msec: 100 mode: - randread norandommap: true op_size: - 4096 osd_ra: - 4096 pool_profile: rbd procs_per_volume: - 1 time: 60 vol_size: 4096 volumes_per_client: - 2 cluster: iterations: 1 osds_per_node: 3 pool_profiles: rbd: pg_size: 128 pgp_size: 128 replication: 3 user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_20:56:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T21:09:19.158 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa; will attempt to use it 2024-09-21T21:09:19.159 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa/tasks 2024-09-21T21:09:19.159 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T21:09:19.161 INFO:teuthology.task.internal:Checking packages... 2024-09-21T21:09:19.181 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '62da54b4b83a96ca9976b7e1d0c8272a564a2208' 2024-09-21T21:09:19.181 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T21:09:19.181 INFO:teuthology.packaging:ref: None 2024-09-21T21:09:19.181 INFO:teuthology.packaging:tag: None 2024-09-21T21:09:19.181 INFO:teuthology.packaging:branch: main 2024-09-21T21:09:19.181 INFO:teuthology.packaging:sha1: 62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-21T21:09:19.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=main 2024-09-21T21:09:19.405 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5092.g62da54b4 2024-09-21T21:09:19.406 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T21:09:19.412 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T21:09:19.412 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T21:09:19.419 INFO:teuthology.task.internal:Saving configuration 2024-09-21T21:09:19.435 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T21:09:19.440 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T21:09:19.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914330', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 21:04:35.152026', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP4HkVd7XU6TAfrQnDFwZO2s5In9BRKqbdvgDi6RoPsjWkjQkQdhZRE0sNwlvEpKJYnqxZV8m8wNVZQVHDjSmAc='} 2024-09-21T21:09:19.467 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T21:09:19.472 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2024-09-21T21:09:19.472 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T21:09:19.502 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f21857ec160>, signals=[15]) 2024-09-21T21:09:19.503 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T21:09:19.509 INFO:teuthology.task.internal:Opening connections... 2024-09-21T21:09:19.509 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-09-21T21:09:19.510 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:09:19.587 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T21:09:19.592 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-09-21T21:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-09-21T21:09:19.630 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-09-21T21:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T21:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T21:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T21:09:19.688 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-09-21T21:09:19.706 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T21:09:19.714 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T21:09:19.719 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T21:09:19.719 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-09-21T21:09:19.743 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T21:09:19.750 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T21:09:19.750 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-09-21T21:09:19.810 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T21:09:19.839 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T21:09:19.853 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T21:09:19.853 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T21:09:19.853 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T21:09:19.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:09:19.870 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-21T21:09:19.927 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-503.el9.x86_64 2024-09-21T21:09:19.927 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-503.el9.x86_64 2024-09-21T21:09:19.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-09-21T21:09:22.466 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T21:09:24.408 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-21T21:09:28.734 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-21T21:09:31.093 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-21T21:09:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-21T21:09:38.261 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T21:09:40.235 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T21:09:40.235 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T21:09:40.336 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:09:40.346 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-09-21T21:09:40.347 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:09:40.348 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 79 M 2024-09-21T21:09:40.348 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 126 M 2024-09-21T21:09:40.348 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:09:40.463 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 386 kB/s | 36 kB 00:00 2024-09-21T21:09:40.730 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-09-21T21:09:41.330 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 33 MB/s | 29 MB 00:00 2024-09-21T21:09:41.655 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-21T21:09:41.656 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:09:41.656 INFO:teuthology.orchestra.run.smithi040.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-21T21:09:42.363 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:09:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:09:42.759 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:09:43.745 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:09:43.745 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:09:46.304 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:09:46.958 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:09:47.048 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:09:49.825 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:09:49.950 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:09:55.072 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:09:55.111 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:39.311 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:39.352 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:39.694 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:39.694 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:10:39.694 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:10:39.694 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:10:40.582 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:10:40.583 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:10:40.920 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:10:41.016 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:10:41.016 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:10:41.016 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T21:10:41.016 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T21:10:41.016 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T21:10:41.016 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T21:10:41.016 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T21:10:41.017 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:10:41.075 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:10:41.075 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:10:41.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2024-09-21T21:10:41.106 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 82.el9 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8274670 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2024-09-21T21:10:41.107 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:10:41.108 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T21:10:41.109 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T21:10:41.109 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 5.14.0-508.el9.x86_64 2024-09-21T21:10:41.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T21:10:41.783 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-09-21T21:10:43.142 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T21:10:43.174 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-09-21T21:10:43.176 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-09-21T21:10:43.191 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.mwFtsKrZ8e 2024-09-21T21:10:43.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mwFtsKrZ8e 2024-09-21T21:10:43.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.mwFtsKrZ8e 2024-09-21T21:10:43.416 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.mwFtsKrZ8e is 6KB 2024-09-21T21:10:43.477 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.mwFtsKrZ8e 2024-09-21T21:10:43.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T21:10:43.572 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T21:10:43.572 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T21:10:43.573 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T21:10:43.573 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T21:10:43.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T21:10:43.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-09-21T21:11:13.741 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T21:11:13.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:11:13.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:11:32.079 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-09-21T21:11:41.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:11:41.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:12:41.139 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T21:12:53.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:12:53.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:12:53.431 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-21T21:12:53.822 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:12:53.823 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T21:12:53.823 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-21T21:12:53.840 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-508.el9.x86_64 2024-09-21T21:12:53.840 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T21:12:53.840 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T21:12:53.840 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T21:12:54.842 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T21:12:54.842 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T21:12:54.907 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-09-21T21:12:54.925 DEBUG:teuthology.parallel:result is None 2024-09-21T21:12:54.925 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T21:12:54.932 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T21:12:54.932 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T21:12:54.950 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T21:12:54.997 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T21:12:55.007 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T21:12:55.007 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T21:12:55.037 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T21:12:55.043 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T21:12:55.043 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T21:12:55.106 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:12:55.118 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:12:55.120 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T21:12:55.126 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T21:12:55.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T21:12:55.189 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T21:12:55.198 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T21:12:55.198 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T21:12:55.247 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:12:55.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:12:55.382 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:12:55.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:12:55.508 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:12:55.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T21:12:55.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-09-21T21:12:55.657 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:12:55.950 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T21:12:55.957 INFO:teuthology.task.internal:Starting timer... 2024-09-21T21:12:55.957 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T21:12:55.970 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T21:12:55.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-09-21T21:12:56.039 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-09-21T21:12:56.041 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T21:12:56.041 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-09-21T21:12:56.062 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-09-21T21:12:56.062 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive'} 2024-09-21T21:12:56.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T21:12:56.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:12:56.130 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T21:12:56.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-09-21T21:12:56.203 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T21:12:56.212 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T21:12:56.222 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T21:12:56.222 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T21:16:45.847 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com')] 2024-09-21T21:16:45.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:16:45.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:16:45.928 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-21T21:16:46.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-21T21:16:46.002 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T21:16:46.020 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T21:16:46.021 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T21:16:46.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:16:46.072 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T21:16:46.088 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T21:16:46.114 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-09-21T21:16:46.124 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-09-21T21:16:46.139 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T21:16:46.154 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T21:16:46.209 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:16:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:16:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-09-21T21:16:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:16:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:16:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:16:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:16:46.214 INFO:teuthology.run_tasks:Running task install... 2024-09-21T21:16:46.223 DEBUG:teuthology.task.install:project ceph 2024-09-21T21:16:46.223 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}} 2024-09-21T21:16:46.223 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'} 2024-09-21T21:16:46.223 INFO:teuthology.task.install:Using flavor: crimson 2024-09-21T21:16:46.228 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T21:16:46.228 INFO:teuthology.task.install:extra packages: [] 2024-09-21T21:16:46.229 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'tag': None, 'wait_for_package': False} 2024-09-21T21:16:46.229 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-21T21:16:46.373 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/ 2024-09-21T21:16:46.373 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5092.g62da54b4 2024-09-21T21:16:46.484 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/62da54b4b83a96ca9976b7e1d0c8272a564a2208/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T21:16:46.484 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:16:46.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T21:16:46.511 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T21:16:46.511 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/62da54b4b83a96ca9976b7e1d0c8272a564a2208/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T21:16:46.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T21:16:46.654 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T21:16:46.719 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-09-21T21:16:46.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-09-21T21:16:46.961 INFO:teuthology.orchestra.run.smithi040.stdout:45 files removed 2024-09-21T21:16:46.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw 2024-09-21T21:16:47.552 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 333 kB/s | 88 kB 00:00 2024-09-21T21:16:47.853 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T21:16:48.114 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 9.4 kB/s | 2.1 kB 00:00 2024-09-21T21:16:48.639 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T21:16:51.915 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 20 MB 00:02 2024-09-21T21:16:56.394 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T21:16:58.192 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-21T21:16:59.271 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T21:17:04.928 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-21T21:17:07.256 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:17:07.269 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 10 M 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.8 M 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T21:17:07.270 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.1 M 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 20 M 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 25 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 808 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T21:17:07.271 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 558 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.4 M 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 45 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 148 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 159 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 314 k 2024-09-21T21:17:07.272 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 302 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 99 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Install 31 Packages 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2024-09-21T21:17:07.273 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:07.274 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 60 M 2024-09-21T21:17:07.274 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:17:07.904 INFO:teuthology.orchestra.run.smithi040.stdout:(1/33): ceph-base-19.3.0-5092.g62da54b4.el9.x86 12 MB/s | 6.1 MB 00:00 2024-09-21T21:17:07.953 INFO:teuthology.orchestra.run.smithi040.stdout:(2/33): ceph-selinux-19.3.0-5092.g62da54b4.el9. 507 kB/s | 25 kB 00:00 2024-09-21T21:17:08.046 INFO:teuthology.orchestra.run.smithi040.stdout:(3/33): libcephfs2-19.3.0-5092.g62da54b4.el9.x8 8.6 MB/s | 808 kB 00:00 2024-09-21T21:17:08.146 INFO:teuthology.orchestra.run.smithi040.stdout:(4/33): libradosstriper1-19.3.0-5092.g62da54b4. 5.4 MB/s | 558 kB 00:00 2024-09-21T21:17:08.296 INFO:teuthology.orchestra.run.smithi040.stdout:(5/33): ceph-radosgw-19.3.0-5092.g62da54b4.el9. 11 MB/s | 10 MB 00:00 2024-09-21T21:17:08.505 INFO:teuthology.orchestra.run.smithi040.stdout:(6/33): ceph-common-19.3.0-5092.g62da54b4.el9.x 18 MB/s | 20 MB 00:01 2024-09-21T21:17:08.530 INFO:teuthology.orchestra.run.smithi040.stdout:(7/33): python3-ceph-argparse-19.3.0-5092.g62da 193 kB/s | 45 kB 00:00 2024-09-21T21:17:08.555 INFO:teuthology.orchestra.run.smithi040.stdout:(8/33): python3-ceph-common-19.3.0-5092.g62da54 2.9 MB/s | 148 kB 00:00 2024-09-21T21:17:08.580 INFO:teuthology.orchestra.run.smithi040.stdout:(9/33): python3-cephfs-19.3.0-5092.g62da54b4.el 3.1 MB/s | 159 kB 00:00 2024-09-21T21:17:08.606 INFO:teuthology.orchestra.run.smithi040.stdout:(10/33): python3-rados-19.3.0-5092.g62da54b4.el 6.1 MB/s | 314 kB 00:00 2024-09-21T21:17:08.631 INFO:teuthology.orchestra.run.smithi040.stdout:(11/33): python3-rbd-19.3.0-5092.g62da54b4.el9. 5.9 MB/s | 302 kB 00:00 2024-09-21T21:17:08.656 INFO:teuthology.orchestra.run.smithi040.stdout:(12/33): python3-rgw-19.3.0-5092.g62da54b4.el9. 1.9 MB/s | 99 kB 00:00 2024-09-21T21:17:08.748 INFO:teuthology.orchestra.run.smithi040.stdout:(13/33): librgw2-19.3.0-5092.g62da54b4.el9.x86_ 8.9 MB/s | 5.4 MB 00:00 2024-09-21T21:17:08.832 INFO:teuthology.orchestra.run.smithi040.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 533 kB/s | 107 kB 00:00 2024-09-21T21:17:08.882 INFO:teuthology.orchestra.run.smithi040.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 756 kB/s | 171 kB 00:00 2024-09-21T21:17:08.908 INFO:teuthology.orchestra.run.smithi040.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-21T21:17:08.933 INFO:teuthology.orchestra.run.smithi040.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-09-21T21:17:08.983 INFO:teuthology.orchestra.run.smithi040.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-21T21:17:09.008 INFO:teuthology.orchestra.run.smithi040.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-09-21T21:17:09.034 INFO:teuthology.orchestra.run.smithi040.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T21:17:09.059 INFO:teuthology.orchestra.run.smithi040.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-21T21:17:09.084 INFO:teuthology.orchestra.run.smithi040.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-21T21:17:09.118 INFO:teuthology.orchestra.run.smithi040.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 563 kB/s | 47 kB 00:00 2024-09-21T21:17:09.177 INFO:teuthology.orchestra.run.smithi040.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-21T21:17:09.202 INFO:teuthology.orchestra.run.smithi040.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-21T21:17:09.227 INFO:teuthology.orchestra.run.smithi040.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-09-21T21:17:09.253 INFO:teuthology.orchestra.run.smithi040.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T21:17:09.286 INFO:teuthology.orchestra.run.smithi040.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-21T21:17:09.345 INFO:teuthology.orchestra.run.smithi040.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T21:17:09.420 INFO:teuthology.orchestra.run.smithi040.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T21:17:09.495 INFO:teuthology.orchestra.run.smithi040.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T21:17:09.579 INFO:teuthology.orchestra.run.smithi040.stdout:(32/33): librados2-19.3.0-5092.g62da54b4.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-21T21:17:09.696 INFO:teuthology.orchestra.run.smithi040.stdout:(33/33): librbd1-19.3.0-5092.g62da54b4.el9.x86_ 14 MB/s | 3.8 MB 00:00 2024-09-21T21:17:09.701 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:17:09.701 INFO:teuthology.orchestra.run.smithi040.stdout:Total 25 MB/s | 60 MB 00:02 2024-09-21T21:17:09.821 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:17:09.862 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:17:09.863 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:17:10.557 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:17:10.557 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:17:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:17:12.668 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-21T21:17:12.757 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-21T21:17:12.849 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-21T21:17:12.950 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-21T21:17:12.974 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-21T21:17:13.037 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-21T21:17:13.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-21T21:17:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-21T21:17:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-21T21:17:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-21T21:17:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-21T21:17:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-21T21:17:13.377 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-21T21:17:13.412 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-21T21:17:13.456 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 11/35 2024-09-21T21:17:13.485 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-21T21:17:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-21T21:17:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-21T21:17:13.687 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 14/35 2024-09-21T21:17:13.723 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-21T21:17:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-21T21:17:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-21T21:17:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-21T21:17:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-21T21:17:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-21T21:17:14.013 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-21T21:17:14.388 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-21T21:17:14.447 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-21T21:17:14.539 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 23/35 2024-09-21T21:17:14.569 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 24/35 2024-09-21T21:17:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-21T21:17:15.248 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-21T21:17:15.275 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-21T21:17:15.347 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 27/35 2024-09-21T21:17:15.440 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 28/35 2024-09-21T21:17:15.470 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 29/35 2024-09-21T21:17:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-21T21:17:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-21T21:17:18.861 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 30/35 2024-09-21T21:17:18.902 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-21T21:17:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 31/35 2024-09-21T21:17:18.989 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T21:17:18.989 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T21:17:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:19.028 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-21T21:17:29.863 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-09-21T21:17:29.864 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-21T21:17:31.041 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 33/35 2024-09-21T21:17:31.041 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:17:31.041 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T21:17:31.041 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:17:31.042 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:17:31.042 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:31.043 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T21:17:31.150 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T21:17:31.150 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 1/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 3/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 4/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 5/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_6 6/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 7/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9. 8/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x8 9/35 2024-09-21T21:17:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 10/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 11/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 12/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 13/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-21T21:17:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-21T21:17:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 32/35 2024-09-21T21:17:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-21T21:17:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 34/35 2024-09-21T21:17:33.515 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T21:17:33.516 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.517 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:33.518 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:17:33.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test 2024-09-21T21:17:34.472 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:30 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:17:34.615 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 81 M 2024-09-21T21:17:34.616 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 175 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:34.617 INFO:teuthology.orchestra.run.smithi040.stdout:Install 9 Packages 2024-09-21T21:17:34.618 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:34.618 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 83 M 2024-09-21T21:17:34.618 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 775 M 2024-09-21T21:17:34.618 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:17:35.281 INFO:teuthology.orchestra.run.smithi040.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 632 kB/s | 94 kB 00:00 2024-09-21T21:17:35.372 INFO:teuthology.orchestra.run.smithi040.stdout:(2/9): libcephsqlite-19.3.0-5092.g62da54b4.el9. 727 kB/s | 175 kB 00:00 2024-09-21T21:17:35.481 INFO:teuthology.orchestra.run.smithi040.stdout:(3/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 2.5 MB/s | 513 kB 00:00 2024-09-21T21:17:35.556 INFO:teuthology.orchestra.run.smithi040.stdout:(4/9): libubsan-11.5.0-2.el9.x86_64.rpm 995 kB/s | 183 kB 00:00 2024-09-21T21:17:35.590 INFO:teuthology.orchestra.run.smithi040.stdout:(5/9): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T21:17:35.640 INFO:teuthology.orchestra.run.smithi040.stdout:(6/9): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-21T21:17:35.657 INFO:teuthology.orchestra.run.smithi040.stdout:(7/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 953 kB/s | 64 kB 00:00 2024-09-21T21:17:35.716 INFO:teuthology.orchestra.run.smithi040.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.6 MB/s | 122 kB 00:00 2024-09-21T21:17:37.641 INFO:teuthology.orchestra.run.smithi040.stdout:(9/9): ceph-test-19.3.0-5092.g62da54b4.el9.x86_ 32 MB/s | 81 MB 00:02 2024-09-21T21:17:37.644 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:17:37.644 INFO:teuthology.orchestra.run.smithi040.stdout:Total 27 MB/s | 83 MB 00:03 2024-09-21T21:17:37.730 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:17:37.756 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:17:37.756 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:17:38.380 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:17:38.381 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:17:39.277 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:17:39.458 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-21T21:17:39.515 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-21T21:17:39.563 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-21T21:17:39.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-21T21:17:39.620 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-21T21:17:39.644 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-21T21:17:39.714 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-21T21:17:39.754 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-21T21:17:39.788 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-21T21:17:51.229 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 8/9 2024-09-21T21:17:51.315 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 9/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 1/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-21T21:17:57.521 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-21T21:17:57.522 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-21T21:17:57.946 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-21T21:17:57.946 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:57.946 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:57.947 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:17:58.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph 2024-09-21T21:17:58.909 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:17:59.052 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:17:59.054 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:17:59.054 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 6.5 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 2.2 M 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 1.6 M 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 292 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 5.1 M 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 70 M 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T21:17:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T21:17:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:59.057 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout:Install 37 Packages 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 84 M 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 896 M 2024-09-21T21:17:59.058 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:17:59.642 INFO:teuthology.orchestra.run.smithi040.stdout:(1/37): ceph-19.3.0-5092.g62da54b4.el9.x86_64.r 29 kB/s | 6.5 kB 00:00 2024-09-21T21:18:00.209 INFO:teuthology.orchestra.run.smithi040.stdout:(2/37): ceph-mgr-19.3.0-5092.g62da54b4.el9.x86_ 2.0 MB/s | 1.6 MB 00:00 2024-09-21T21:18:00.334 INFO:teuthology.orchestra.run.smithi040.stdout:(3/37): ceph-mon-19.3.0-5092.g62da54b4.el9.x86_ 7.4 MB/s | 5.1 MB 00:00 2024-09-21T21:18:00.384 INFO:teuthology.orchestra.run.smithi040.stdout:(4/37): ceph-mds-19.3.0-5092.g62da54b4.el9.x86_ 2.3 MB/s | 2.2 MB 00:00 2024-09-21T21:18:00.410 INFO:teuthology.orchestra.run.smithi040.stdout:(5/37): ceph-mgr-modules-core-19.3.0-5092.g62da 3.8 MB/s | 292 kB 00:00 2024-09-21T21:18:01.160 INFO:teuthology.orchestra.run.smithi040.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-21T21:18:01.343 INFO:teuthology.orchestra.run.smithi040.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 1.3 MB/s | 1.2 MB 00:00 2024-09-21T21:18:01.827 INFO:teuthology.orchestra.run.smithi040.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-21T21:18:02.528 INFO:teuthology.orchestra.run.smithi040.stdout:(9/37): ceph-osd-19.3.0-5092.g62da54b4.el9.x86_ 30 MB/s | 70 MB 00:02 2024-09-21T21:18:02.553 INFO:teuthology.orchestra.run.smithi040.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 112 kB/s | 135 kB 00:01 2024-09-21T21:18:02.678 INFO:teuthology.orchestra.run.smithi040.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-21T21:18:02.728 INFO:teuthology.orchestra.run.smithi040.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T21:18:02.770 INFO:teuthology.orchestra.run.smithi040.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 839 kB/s | 35 kB 00:00 2024-09-21T21:18:02.837 INFO:teuthology.orchestra.run.smithi040.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-21T21:18:02.871 INFO:teuthology.orchestra.run.smithi040.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-21T21:18:02.904 INFO:teuthology.orchestra.run.smithi040.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.3 MB/s | 42 kB 00:00 2024-09-21T21:18:02.963 INFO:teuthology.orchestra.run.smithi040.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 384 kB/s | 22 kB 00:00 2024-09-21T21:18:03.021 INFO:teuthology.orchestra.run.smithi040.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T21:18:03.047 INFO:teuthology.orchestra.run.smithi040.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-21T21:18:03.080 INFO:teuthology.orchestra.run.smithi040.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-09-21T21:18:03.122 INFO:teuthology.orchestra.run.smithi040.stdout:(21/37): python3-requests-2.25.1-8.el9.noarch.r 96 kB/s | 125 kB 00:01 2024-09-21T21:18:03.173 INFO:teuthology.orchestra.run.smithi040.stdout:(22/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.8 MB/s | 358 kB 00:00 2024-09-21T21:18:03.190 INFO:teuthology.orchestra.run.smithi040.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-09-21T21:18:03.206 INFO:teuthology.orchestra.run.smithi040.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 529 kB/s | 18 kB 00:00 2024-09-21T21:18:03.223 INFO:teuthology.orchestra.run.smithi040.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 693 kB/s | 23 kB 00:00 2024-09-21T21:18:03.249 INFO:teuthology.orchestra.run.smithi040.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 461 kB/s | 19 kB 00:00 2024-09-21T21:18:03.266 INFO:teuthology.orchestra.run.smithi040.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 467 kB/s | 20 kB 00:00 2024-09-21T21:18:03.283 INFO:teuthology.orchestra.run.smithi040.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-21T21:18:03.308 INFO:teuthology.orchestra.run.smithi040.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-21T21:18:03.325 INFO:teuthology.orchestra.run.smithi040.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 996 kB/s | 16 kB 00:00 2024-09-21T21:18:03.350 INFO:teuthology.orchestra.run.smithi040.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T21:18:03.376 INFO:teuthology.orchestra.run.smithi040.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-21T21:18:03.401 INFO:teuthology.orchestra.run.smithi040.stdout:(33/37): python3-urllib3-1.26.5-6.el9.noarch.rp 246 kB/s | 215 kB 00:00 2024-09-21T21:18:03.418 INFO:teuthology.orchestra.run.smithi040.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-09-21T21:18:03.452 INFO:teuthology.orchestra.run.smithi040.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T21:18:03.468 INFO:teuthology.orchestra.run.smithi040.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 474 kB/s | 20 kB 00:00 2024-09-21T21:18:03.502 INFO:teuthology.orchestra.run.smithi040.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-21T21:18:03.508 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:03.508 INFO:teuthology.orchestra.run.smithi040.stdout:Total 19 MB/s | 84 MB 00:04 2024-09-21T21:18:03.666 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:03.713 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:03.713 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:04.536 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:04.536 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:06.254 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:06.313 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T21:18:06.346 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T21:18:06.374 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T21:18:06.448 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T21:18:06.498 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T21:18:06.531 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T21:18:06.638 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T21:18:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T21:18:06.769 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T21:18:06.812 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T21:18:06.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T21:18:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T21:18:06.909 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T21:18:06.958 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T21:18:07.002 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T21:18:07.036 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T21:18:07.085 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T21:18:07.134 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T21:18:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T21:18:07.371 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T21:18:07.453 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T21:18:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T21:18:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T21:18:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T21:18:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T21:18:07.869 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T21:18:07.912 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T21:18:07.993 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T21:18:08.098 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T21:18:08.155 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T21:18:08.296 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T21:18:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 32/37 2024-09-21T21:18:08.529 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-21T21:18:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 33/37 2024-09-21T21:18:08.556 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:18:08.556 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T21:18:08.556 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:18:08.557 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:18:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 34/37 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:18:19.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:20.322 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 35/37 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:18:20.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:20.598 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 36/37 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:18:20.623 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:20.698 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-21T21:18:27.157 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 37/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 1/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 3/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 4/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 5/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9. 6/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T21:18:27.158 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T21:18:27.159 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T21:18:27.160 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T21:18:27.160 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T21:18:27.160 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T21:18:27.160 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T21:18:27.161 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T21:18:28.089 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T21:18:28.090 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T21:18:28.091 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:28.092 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:28.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base 2024-09-21T21:18:28.985 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:29.078 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:18:29.122 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:18:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:29.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install cephadm 2024-09-21T21:18:29.799 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:29.935 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 784 k 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:18:29.936 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:29.937 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:18:29.937 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:29.937 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 784 k 2024-09-21T21:18:29.937 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 791 k 2024-09-21T21:18:29.937 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:18:30.439 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm-19.3.0-5092.g62da54b4.el9.noarch.rpm 1.5 MB/s | 784 kB 00:00 2024-09-21T21:18:30.439 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:30.439 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.5 MB/s | 784 kB 00:00 2024-09-21T21:18:30.440 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:30.446 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:30.446 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:30.452 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:30.452 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:30.529 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:18:31.090 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:18:31.449 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:31.800 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:32.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T21:18:32.654 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:32.792 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 158 k 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 158 k 2024-09-21T21:18:32.793 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 644 k 2024-09-21T21:18:32.794 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:18:32.996 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-19.3.0-5092.g62da54 780 kB/s | 158 kB 00:00 2024-09-21T21:18:32.996 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:32.997 INFO:teuthology.orchestra.run.smithi040.stdout:Total 775 kB/s | 158 kB 00:00 2024-09-21T21:18:32.997 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:33.008 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:33.008 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:33.069 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:33.069 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:33.222 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:33.372 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-21T21:18:33.400 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-21T21:18:33.400 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:18:33.400 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T21:18:33.400 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4. 1/1 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:34.310 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:34.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr 2024-09-21T21:18:35.172 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:35.263 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-mgr-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:18:35.308 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:35.309 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:18:35.309 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:35.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T21:18:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:36.133 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 14 M 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:18:36.134 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 34 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 16 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:18:36.135 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:36.136 INFO:teuthology.orchestra.run.smithi040.stdout:Install 11 Packages 2024-09-21T21:18:36.136 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:36.136 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 18 M 2024-09-21T21:18:36.136 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 156 M 2024-09-21T21:18:36.136 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:18:36.633 INFO:teuthology.orchestra.run.smithi040.stdout:(1/11): ceph-prometheus-alerts-19.3.0-5092.g62d 103 kB/s | 16 kB 00:00 2024-09-21T21:18:36.666 INFO:teuthology.orchestra.run.smithi040.stdout:(2/11): ceph-grafana-dashboards-19.3.0-5092.g62 178 kB/s | 34 kB 00:00 2024-09-21T21:18:36.833 INFO:teuthology.orchestra.run.smithi040.stdout:(3/11): protobuf-compiler-3.14.0-13.el9.x86_64. 5.1 MB/s | 863 kB 00:00 2024-09-21T21:18:36.859 INFO:teuthology.orchestra.run.smithi040.stdout:(4/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-21T21:18:36.909 INFO:teuthology.orchestra.run.smithi040.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 459 kB/s | 19 kB 00:00 2024-09-21T21:18:36.951 INFO:teuthology.orchestra.run.smithi040.stdout:(6/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-21T21:18:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-21T21:18:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-21T21:18:37.035 INFO:teuthology.orchestra.run.smithi040.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-21T21:18:37.110 INFO:teuthology.orchestra.run.smithi040.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-09-21T21:18:37.269 INFO:teuthology.orchestra.run.smithi040.stdout:(11/11): ceph-mgr-dashboard-19.3.0-5092.g62da54 18 MB/s | 14 MB 00:00 2024-09-21T21:18:37.273 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:37.273 INFO:teuthology.orchestra.run.smithi040.stdout:Total 16 MB/s | 18 MB 00:01 2024-09-21T21:18:37.304 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:37.324 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:37.325 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:37.604 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:37.604 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:38.171 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-21T21:18:38.281 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-21T21:18:38.386 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-21T21:18:38.522 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-21T21:18:38.585 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-21T21:18:38.673 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-21T21:18:38.721 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-21T21:18:38.748 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-21T21:18:38.781 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 9/11 2024-09-21T21:18:42.238 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 10/11 2024-09-21T21:18:42.293 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 11/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el 1/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noa 2/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9 3/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-21T21:18:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-21T21:18:44.466 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-21T21:18:44.466 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:44.466 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:44.467 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:44.704 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T21:18:45.316 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:45.456 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 7.4 M 2024-09-21T21:18:45.457 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T21:18:45.458 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout:======================================================================================= 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout:Install 12 Packages 2024-09-21T21:18:45.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:45.460 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 38 M 2024-09-21T21:18:45.460 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 212 M 2024-09-21T21:18:45.460 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:18:46.084 INFO:teuthology.orchestra.run.smithi040.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 718 kB/s | 191 kB 00:00 2024-09-21T21:18:46.117 INFO:teuthology.orchestra.run.smithi040.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-21T21:18:46.192 INFO:teuthology.orchestra.run.smithi040.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-21T21:18:46.293 INFO:teuthology.orchestra.run.smithi040.stdout:(4/12): ceph-mgr-diskprediction-local-19.3.0-50 16 MB/s | 7.4 MB 00:00 2024-09-21T21:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-09-21T21:18:46.393 INFO:teuthology.orchestra.run.smithi040.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-21T21:18:46.510 INFO:teuthology.orchestra.run.smithi040.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-09-21T21:18:46.602 INFO:teuthology.orchestra.run.smithi040.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 17 MB/s | 4.9 MB 00:00 2024-09-21T21:18:46.652 INFO:teuthology.orchestra.run.smithi040.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-21T21:18:46.711 INFO:teuthology.orchestra.run.smithi040.stdout:(10/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 5.0 MB/s | 3.0 MB 00:00 2024-09-21T21:18:46.803 INFO:teuthology.orchestra.run.smithi040.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-21T21:18:47.078 INFO:teuthology.orchestra.run.smithi040.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-21T21:18:47.079 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:47.080 INFO:teuthology.orchestra.run.smithi040.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-21T21:18:47.445 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:47.474 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:47.474 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:47.919 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:47.919 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:48.810 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T21:18:48.852 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:18:48.955 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:18:48.995 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T21:18:49.187 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T21:18:49.218 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T21:18:49.297 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T21:18:49.988 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T21:18:50.104 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T21:18:51.425 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:18:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T21:18:52.535 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-21T21:18:55.109 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 12/12 2024-09-21T21:18:55.109 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da5 1/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:18:55.110 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T21:18:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T21:18:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T21:18:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T21:18:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T21:18:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:55.499 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:18:55.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-rook 2024-09-21T21:18:56.517 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:52 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:18:56.658 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:18:56.659 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:56.659 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:18:56.659 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 49 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T21:18:56.660 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout:Install 13 Packages 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:18:56.661 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.2 M 2024-09-21T21:18:56.662 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 26 M 2024-09-21T21:18:56.662 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:18:56.811 INFO:teuthology.orchestra.run.smithi040.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 291 kB/s | 26 kB 00:00 2024-09-21T21:18:56.836 INFO:teuthology.orchestra.run.smithi040.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 169 kB/s | 19 kB 00:00 2024-09-21T21:18:56.895 INFO:teuthology.orchestra.run.smithi040.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-09-21T21:18:56.920 INFO:teuthology.orchestra.run.smithi040.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-21T21:18:56.936 INFO:teuthology.orchestra.run.smithi040.stdout:(5/13): ceph-mgr-rook-19.3.0-5092.g62da54b4.el9 230 kB/s | 49 kB 00:00 2024-09-21T21:18:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.6 MB/s | 54 kB 00:00 2024-09-21T21:18:56.979 INFO:teuthology.orchestra.run.smithi040.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-09-21T21:18:57.021 INFO:teuthology.orchestra.run.smithi040.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 384 kB/s | 32 kB 00:00 2024-09-21T21:18:57.038 INFO:teuthology.orchestra.run.smithi040.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-21T21:18:57.063 INFO:teuthology.orchestra.run.smithi040.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T21:18:57.088 INFO:teuthology.orchestra.run.smithi040.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.3 MB/s | 220 kB 00:00 2024-09-21T21:18:57.114 INFO:teuthology.orchestra.run.smithi040.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-21T21:18:57.147 INFO:teuthology.orchestra.run.smithi040.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 8.2 MB/s | 1.0 MB 00:00 2024-09-21T21:18:57.150 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:18:57.150 INFO:teuthology.orchestra.run.smithi040.stdout:Total 4.5 MB/s | 2.2 MB 00:00 2024-09-21T21:18:57.215 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:18:57.231 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:18:57.231 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:18:57.332 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:18:57.334 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:18:57.853 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:18:58.034 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T21:18:58.092 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T21:18:58.148 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T21:18:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T21:18:58.225 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T21:18:58.331 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T21:18:58.441 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T21:18:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T21:18:59.249 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T21:18:59.299 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T21:18:59.333 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T21:18:59.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T21:18:59.419 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-21T21:19:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 13/13 2024-09-21T21:19:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 1/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T21:19:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T21:19:00.515 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T21:19:00.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T21:19:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T21:19:00.517 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:00.517 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:00.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T21:19:01.406 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:57 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:01.545 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:01.545 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:01.545 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 154 k 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:01.546 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2024-09-21T21:19:01.547 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:01.547 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 7.0 M 2024-09-21T21:19:01.547 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 33 M 2024-09-21T21:19:01.547 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:02.050 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): ceph-mgr-cephadm-19.3.0-5092.g62da54b4.e 725 kB/s | 154 kB 00:00 2024-09-21T21:19:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-21T21:19:02.150 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.6 MB/s | 58 kB 00:00 2024-09-21T21:19:02.175 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-21T21:19:02.617 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 318 kB/s | 248 kB 00:00 2024-09-21T21:19:02.976 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-21T21:19:02.978 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:02.979 INFO:teuthology.orchestra.run.smithi040.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-09-21T21:19:03.055 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:03.068 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:03.068 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:03.177 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:03.177 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:03.495 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:03.667 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T21:19:03.731 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T21:19:04.297 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T21:19:04.383 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T21:19:04.460 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T21:19:04.490 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 6/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 1/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T21:19:05.300 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T21:19:05.598 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T21:19:05.598 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:05.598 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:05.598 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:05.599 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:05.870 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse 2024-09-21T21:19:06.475 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:02 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:06.613 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 883 k 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2024-09-21T21:19:06.614 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:06.615 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 963 k 2024-09-21T21:19:06.615 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 4.1 M 2024-09-21T21:19:06.615 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:06.761 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 819 kB/s | 80 kB 00:00 2024-09-21T21:19:06.961 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-fuse-19.3.0-5092.g62da54b4.el9.x86_ 2.9 MB/s | 883 kB 00:00 2024-09-21T21:19:06.962 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:06.962 INFO:teuthology.orchestra.run.smithi040.stdout:Total 2.7 MB/s | 963 kB 00:00 2024-09-21T21:19:06.974 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:06.999 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:06.999 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:07.086 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:07.086 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:07.277 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:07.436 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T21:19:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-21T21:19:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2/2 2024-09-21T21:19:08.248 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/2 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:08.573 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:08.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-volume 2024-09-21T21:19:09.487 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:05 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:09.624 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:19.3.0-5092.g62da54b4.el9 ceph-noarch 285 k 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:09.625 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 285 k 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.3 M 2024-09-21T21:19:09.626 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:09.850 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-volume-19.3.0-5092.g62da54b4.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-21T21:19:09.851 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:09.851 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-21T21:19:09.851 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:09.860 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:09.860 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:09.890 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:09.891 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:10.122 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:10.185 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:19:10.201 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:19:10.202 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:19:10.202 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T21:19:10.202 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 1/1 2024-09-21T21:19:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:11.229 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:19.3.0-5092.g62da54b4.el9.noarch 2024-09-21T21:19:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:11.229 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:11.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel 2024-09-21T21:19:12.161 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:08 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:12.303 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 125 k 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:12.305 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 125 k 2024-09-21T21:19:12.305 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 445 k 2024-09-21T21:19:12.305 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:12.496 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-19.3.0-5092.g62da54b4.el9.x86_64 654 kB/s | 125 kB 00:00 2024-09-21T21:19:12.497 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:12.497 INFO:teuthology.orchestra.run.smithi040.stdout:Total 647 kB/s | 125 kB 00:00 2024-09-21T21:19:12.497 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:12.503 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:12.503 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:12.571 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:12.571 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:12.733 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:12.830 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:13.555 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:13.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2 2024-09-21T21:19:14.398 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:10 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:14.492 INFO:teuthology.orchestra.run.smithi040.stdout:Package libcephfs2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:14.539 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:14.540 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:14.540 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:14.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs-devel 2024-09-21T21:19:15.216 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:11 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:15.355 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 32 k 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:15.356 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:15.357 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 32 k 2024-09-21T21:19:15.357 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 142 k 2024-09-21T21:19:15.357 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:15.500 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs-devel-19.3.0-5092.g62da54b4.el9.x86_6 222 kB/s | 32 kB 00:00 2024-09-21T21:19:15.500 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:15.500 INFO:teuthology.orchestra.run.smithi040.stdout:Total 220 kB/s | 32 kB 00:00 2024-09-21T21:19:15.501 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:15.505 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:15.505 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:15.520 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:15.521 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:15.609 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:16.123 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:16.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:16.370 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:16.370 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:16.370 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:19:16.370 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:16.370 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:16.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2 2024-09-21T21:19:17.167 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:13 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:17.259 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:17.305 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:17.306 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:17.306 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:17.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1 2024-09-21T21:19:17.990 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:13 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:18.083 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:18.129 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:18.130 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:18.130 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:18.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados 2024-09-21T21:19:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:14 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:18.902 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rados-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:18.949 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:18.950 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:18.950 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:19.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw 2024-09-21T21:19:19.629 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:15 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:19.721 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rgw-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:19.769 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:19.770 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:19.770 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:19.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs 2024-09-21T21:19:20.458 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:16 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:20.550 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-cephfs-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:20.596 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:20.597 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:20.597 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd 2024-09-21T21:19:21.280 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:17 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:21.372 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rbd-2:19.3.0-5092.g62da54b4.el9.x86_64 is already installed. 2024-09-21T21:19:21.419 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:21.420 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-21T21:19:21.420 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:21.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse 2024-09-21T21:19:22.106 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:18 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:22.243 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:22.244 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:22.244 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 91 k 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:22.245 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:22.246 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:22.246 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:22.246 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 91 k 2024-09-21T21:19:22.246 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 321 k 2024-09-21T21:19:22.246 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:22.445 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-19.3.0-5092.g62da54b4.el9.x86_64.rpm 455 kB/s | 91 kB 00:00 2024-09-21T21:19:22.446 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:22.446 INFO:teuthology.orchestra.run.smithi040.stdout:Total 452 kB/s | 91 kB 00:00 2024-09-21T21:19:22.446 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:22.455 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:22.455 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:22.523 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:22.523 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:22.779 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:23.462 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:23.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror 2024-09-21T21:19:24.444 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:20 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:24.588 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 3.5 M 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:24.589 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 3.5 M 2024-09-21T21:19:24.590 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 30 M 2024-09-21T21:19:24.590 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:24.915 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-19.3.0-5092.g62da54b4.el9.x86_64.rpm 11 MB/s | 3.5 MB 00:00 2024-09-21T21:19:24.915 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:24.916 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 3.5 MB 00:00 2024-09-21T21:19:24.916 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:24.928 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:24.928 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:25.018 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:25.018 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:25.640 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:25.707 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:25.734 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:25.734 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:19:25.734 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T21:19:25.735 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:19:25.735 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:19:25.735 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:27.050 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:27.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd 2024-09-21T21:19:27.940 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:02:23 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 176 k 2024-09-21T21:19:28.079 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 176 k 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 685 k 2024-09-21T21:19:28.080 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:19:28.293 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-19.3.0-5092.g62da54b4.el9.x86_64.rpm 828 kB/s | 176 kB 00:00 2024-09-21T21:19:28.294 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:19:28.294 INFO:teuthology.orchestra.run.smithi040.stdout:Total 823 kB/s | 176 kB 00:00 2024-09-21T21:19:28.294 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:19:28.305 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:19:28.306 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:19:28.374 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:19:28.374 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:19:28.537 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:19:28.636 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:29.104 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 1/1 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:29.378 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:19:29.630 DEBUG:teuthology.parallel:result is None 2024-09-21T21:19:29.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=62da54b4b83a96ca9976b7e1d0c8272a564a2208 2024-09-21T21:19:29.777 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T21:19:29.799 INFO:teuthology.orchestra.run.smithi040.stdout:19.3.0-5092.g62da54b4.el9 2024-09-21T21:19:29.799 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5092.g62da54b4.el9 2024-09-21T21:19:29.799 INFO:teuthology.task.install:The correct ceph version 19.3.0-5092.g62da54b4 is installed. 2024-09-21T21:19:29.801 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T21:19:29.801 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:29.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T21:19:29.865 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T21:19:29.865 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:29.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T21:19:29.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T21:19:29.995 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T21:19:29.995 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:29.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T21:19:30.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T21:19:30.127 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T21:19:30.127 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:30.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T21:19:30.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T21:19:30.257 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T21:19:30.336 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T21:19:30.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-09-21T21:19:30.365 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T21:19:30.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T21:19:30.431 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T21:19:30.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T21:19:30.504 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T21:19:30.504 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T21:19:30.504 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi/7914330', 'branch': 'main', 'description': 'crimson-rados/perf/{clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore settings/optimized workloads/fio_4K_rand_read}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914330', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_20:56:02-crimson-rados-main-distro-crimson-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2950, 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_62da54b4b83a96ca9976b7e1d0c8272a564a2208/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP4HkVd7XU6TAfrQnDFwZO2s5In9BRKqbdvgDi6RoPsjWkjQkQdhZRE0sNwlvEpKJYnqxZV8m8wNVZQVHDjSmAc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'wait-for-scrub': False, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': '62da54b4b83a96ca9976b7e1d0c8272a564a2208', 'cluster': 'ceph'}}, {'cbt': {'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randread'], 'norandommap': True, 'op_size': [4096], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2]}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_20:56:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T21:19:30.505 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T21:19:30.554 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T21:19:30.620 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:30.620 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T21:19:30.679 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T21:19:30.679 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-21 21:19:26.011307813 +0000 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-21 21:16:25.677745163 +0000 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-21 21:16:25.677745163 +0000 2024-09-21T21:19:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-21 21:16:25.677745163 +0000 2024-09-21T21:19:30.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T21:19:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-21T21:19:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-21T21:19:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.00020361 s, 2.5 MB/s 2024-09-21T21:19:30.809 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T21:19:30.865 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-21 21:19:26.011307813 +0000 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-21 21:16:26.015741024 +0000 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-21 21:16:26.015741024 +0000 2024-09-21T21:19:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-21 21:16:26.015741024 +0000 2024-09-21T21:19:30.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T21:19:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-21T21:19:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-21T21:19:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.0001853 s, 2.8 MB/s 2024-09-21T21:19:30.987 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T21:19:31.043 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-09-21T21:19:31.099 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T21:19:31.099 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-21 21:19:26.012307801 +0000 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-21 21:16:26.353736886 +0000 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-21 21:16:26.353736886 +0000 2024-09-21T21:19:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-21 21:16:26.353736886 +0000 2024-09-21T21:19:31.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T21:19:31.163 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-21T21:19:31.163 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-21T21:19:31.164 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000227243 s, 2.3 MB/s 2024-09-21T21:19:31.165 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T21:19:31.221 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-21 21:19:26.012307801 +0000 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-21 21:16:26.688732784 +0000 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-21 21:16:26.688732784 +0000 2024-09-21T21:19:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-21 21:16:26.688732784 +0000 2024-09-21T21:19:31.279 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T21:19:31.341 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-21T21:19:31.341 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-21T21:19:31.341 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000229667 s, 2.2 MB/s 2024-09-21T21:19:31.343 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T21:19:31.399 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:19:31.399 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-21T21:19:31.399 INFO:tasks.ceph:Generating config... 2024-09-21T21:19:31.400 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug filestore = 0/0 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug journal = 0/0 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug ms = 0/0 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug osd = 0/0 2024-09-21T21:19:31.401 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] err to stderr = True 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] log flush on exit = True 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] log to file = False 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] log to stderr = True 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T21:19:31.402 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-21T21:19:31.402 INFO:tasks.ceph:[global] auth client required = none 2024-09-21T21:19:31.402 INFO:tasks.ceph:[global] auth cluster required = none 2024-09-21T21:19:31.402 INFO:tasks.ceph:[global] auth service required = none 2024-09-21T21:19:31.402 INFO:tasks.ceph:[global] auth supported = none 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug asok = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug auth = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug bdev = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug bluefs = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug bluestore = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug buffer = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug civetweb = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug client = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug compressor = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug context = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug crush = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug crypto = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug dpdk = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2024-09-21T21:19:31.403 INFO:tasks.ceph:[global] debug filer = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug filestore = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug finisher = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug fuse = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug javaclient = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug journal = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug journaler = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug kstore = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug leveldb = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug lockdep = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds locker = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds log = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2024-09-21T21:19:31.404 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug memdb = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug mgr = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug mgrc = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug mon = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug monc = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug ms = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug objclass = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug objecter = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug optracker = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug osd = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug paxos = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug rados = 0/0 2024-09-21T21:19:31.405 INFO:tasks.ceph:[global] debug rbd = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug refs = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rgw = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug striper = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug throttle = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug timer = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] debug tp = 0/0 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-21T21:19:31.406 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-21T21:19:31.407 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-21T21:19:31.407 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-21T21:19:31.407 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-21T21:19:31.407 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mon] debug mon = 0/0 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mon] debug ms = 0/0 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mon] debug paxos = 0/0 2024-09-21T21:19:31.407 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-21T21:19:31.407 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-09-21T21:19:31.407 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-09-21T21:19:31.407 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-09-21T21:19:31.407 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-09-21T21:19:31.408 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T21:19:31.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T21:19:31.492 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T21:19:31.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T21:19:31.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:19:31.574 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40')] 2024-09-21T21:19:31.574 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-09-21T21:19:31.575 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:31.575 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T21:19:31.630 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:31.723+0000 7f1c70e3cf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:31.723+0000 7f1c70e3cf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:31.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:31.723+0000 7f1c70e3cf00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 9e03ce4a-d956-465f-ae25-1c458f0b4938 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = quincy 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 9e03ce4a-d956-465f-ae25-1c458f0b4938 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-09-21T21:19:31.724453+0000 2024-09-21T21:19:31.725 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-09-21T21:19:31.724453+0000 2024-09-21T21:19:31.726 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 17 (quincy) 2024-09-21T21:19:31.726 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-09-21T21:19:31.726 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-09-21T21:19:31.726 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-21T21:19:31.728 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T21:19:31.743 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9e03ce4a-d956-465f-ae25-1c458f0b4938... 2024-09-21T21:19:31.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = none 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-09-21T21:19:31.829 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-09-21T21:19:31.830 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = 172.21.15.40 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: auth client required = none 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: auth cluster required = none 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: auth service required = none 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug asok = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug bdev = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug buffer = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug civetweb = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 0/0 2024-09-21T21:19:31.832 INFO:teuthology.orchestra.run.smithi040.stdout: debug compressor = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug context = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug crush = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug crypto = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug dpdk = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug eventtrace = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug filer = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug filestore = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug finisher = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug fuse = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug heartbeatmap = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug javaclient = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug journal = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug journaler = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug kstore = 0/0 2024-09-21T21:19:31.833 INFO:teuthology.orchestra.run.smithi040.stdout: debug leveldb = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug lockdep = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds locker = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds log = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds log expire = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds migrator = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug memdb = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgrc = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug monc = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug objclass = 0/0 2024-09-21T21:19:31.834 INFO:teuthology.orchestra.run.smithi040.stdout: debug objectcacher = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug objecter = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug optracker = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug perfcounter = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rados = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rbd = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rbd mirror = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rbd replay = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug refs = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rgw = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rgw lifecycle = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rgw notification = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rgw sync = 0/0 2024-09-21T21:19:31.835 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 0/0 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: debug striper = 0/0 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: debug throttle = 0/0 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: debug timer = 0/0 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: debug tp = 0/0 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms client mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms cluster mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms mon client mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms mon cluster mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms mon service mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: ms service mode = crc 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 9e03ce4a-d956-465f-ae25-1c458f0b4938 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.836 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug trim objects = true 2024-09-21T21:19:31.837 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-09-21T21:19:31.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: debug monc = 20 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore compression algorithm = snappy 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore compression mode = aggressive 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore rocksdb cf = False 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: crimson alien thread cpu cores = 6-7 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: crimson osd obc lru size = 10 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: debug filestore = 0/0 2024-09-21T21:19:31.839 INFO:teuthology.orchestra.run.smithi040.stdout: debug journal = 0/0 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 0/0 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 0/0 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: err to stderr = True 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: log flush on exit = True 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: log to file = False 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: log to stderr = True 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-09-21T21:19:31.840 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: osd shutdown pgref assert = True 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-09-21T21:19:31.841 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 0/0 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 0/0 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 0/0 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.842 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-09-21T21:19:31.843 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default crimson = True 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: crimson seastar cpu cores = 0-2 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: crimson seastar cpu cores = 3-5 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-09-21T21:19:31.844 INFO:teuthology.orchestra.run.smithi040.stdout: crimson seastar cpu cores = 0-2 2024-09-21T21:19:31.845 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-09-21T21:19:31.845 INFO:teuthology.orchestra.run.smithi040.stdout: crimson seastar cpu cores = 3-5 2024-09-21T21:19:31.846 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T21:19:31.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T21:19:31.912 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T21:19:31.912 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:31.912 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T21:19:31.928 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:31.928 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T21:19:31.984 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-09-21T21:19:31.984 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:31.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T21:19:31.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:19:32.059 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:32.059 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:32.115 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T21:19:32.115 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T21:19:32.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T21:19:32.216 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T21:19:32.219 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T21:19:32.219 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T21:19:32.219 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T21:19:32.270 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T21:19:32.284 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T21:19:32.284 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-21T21:19:32.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T21:19:32.310 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:19:32.310 INFO:tasks.ceph:role: osd.0 2024-09-21T21:19:32.310 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-21T21:19:32.310 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T21:19:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:19:32.383 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:19:32.383 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:19:32.383 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:19:32.383 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:19:32.384 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:19:32.384 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:19:32.384 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:19:32.384 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:19:32.384 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:19:32.902 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-21T21:19:32.904 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-21T21:19:32.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T21:19:33.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T21:19:33.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T21:19:33.131 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:19:33.131 INFO:tasks.ceph:role: osd.1 2024-09-21T21:19:33.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-21T21:19:33.131 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T21:19:33.204 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:19:33.205 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:19:33.649 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-21T21:19:33.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-21T21:19:33.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T21:19:33.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T21:19:33.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T21:19:33.870 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:19:33.870 INFO:tasks.ceph:role: osd.2 2024-09-21T21:19:33.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-21T21:19:33.871 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:19:33.943 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:19:34.437 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-21T21:19:34.439 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-21T21:19:34.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T21:19:34.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T21:19:34.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:34,936 seastar - Reactor backend: linux-aio 2024-09-21T21:19:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:==27914==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:35.059 INFO:teuthology.orchestra.run.smithi040.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x3610fad 0x396c001 0x4585f9c 0x4622f00 0x5fbf625 0x5f9c36b 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3961ae8 0x39632c2 0x3803c18 0x2958f 0x2963f 0x35eb9a4 2024-09-21T21:19:35.060 INFO:teuthology.orchestra.run.smithi040.stderr:kernel callstack: 2024-09-21T21:19:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:DEBUG 2024-09-21 21:19:35,094 [shard 0:main] seastar - reactor::drain 2024-09-21T21:19:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:35.629 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:35.629 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:35.629 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:35.630 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:35,688 seastar - Reactor backend: linux-aio 2024-09-21T21:19:35.698 INFO:teuthology.orchestra.run.smithi040.stderr:==27912==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:35.763 INFO:teuthology.orchestra.run.smithi040.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x156cf 0xdd312 0x4ab1c 0x5fccc96 0x5fcce92 0x5fcd08f 0x5fcd365 0x601ba63 0x601bc69 0x5f9bdb6 0x5f9c2ea 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-21T21:19:35.763 INFO:teuthology.orchestra.run.smithi040.stderr:kernel callstack: 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:40.427 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:40.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T21:19:40.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:40,769 seastar - Reactor backend: linux-aio 2024-09-21T21:19:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:==28100==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:40.884 INFO:teuthology.orchestra.run.smithi040.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x5fc0436 0x5fc09aa 0x601bbd4 0x5f9bdb6 0x5f9c2ea 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3961ae8 0x39632c2 0x3803c18 0x2958f 0x2963f 0x35eb9a4 2024-09-21T21:19:40.884 INFO:teuthology.orchestra.run.smithi040.stderr:kernel callstack: 2024-09-21T21:19:40.918 INFO:teuthology.orchestra.run.smithi040.stderr:DEBUG 2024-09-21 21:19:40,917 [shard 0:main] seastar - reactor::drain 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:41.448 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:41,512 seastar - Reactor backend: linux-aio 2024-09-21T21:19:41.522 INFO:teuthology.orchestra.run.smithi040.stderr:==28098==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:46.282 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:46.282 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:46.282 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:46.283 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:46.283 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:46.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T21:19:46.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:46.625 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:46.626 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:46,626 seastar - Reactor backend: linux-aio 2024-09-21T21:19:46.631 INFO:teuthology.orchestra.run.smithi040.stderr:==28286==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x124f5f 0x44735 0xfe5da 0x3626869 0x3626983 0x5f9cb34 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3961ae8 0x39632c2 0x3803c18 0x2958f 0x2963f 0x35eb9a4 2024-09-21T21:19:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:kernel callstack: 2024-09-21T21:19:46.774 INFO:teuthology.orchestra.run.smithi040.stderr:DEBUG 2024-09-21 21:19:46,774 [shard 0:main] seastar - reactor::drain 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:47.303 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:47.358 INFO:teuthology.orchestra.run.smithi040.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-21T21:19:47.362 INFO:teuthology.orchestra.run.smithi040.stderr:INFO 2024-09-21 21:19:47,362 seastar - Reactor backend: linux-aio 2024-09-21T21:19:47.376 INFO:teuthology.orchestra.run.smithi040.stderr:==28284==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-21T21:19:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbc8b6ef 0xbb64079 0xbb655e4 0xbb657fa 0xbb65950 0xbb65e19 0x3e6ef 0x10d76c 0xdd31a 0x4b14c 0x381a15c 0x5fc04d3 0x5fc09aa 0x601bbd4 0x5f9bdb6 0x5f9c2ea 0x5de40ab 0x5dfc2ed 0x713a942 0x714c115 0x714d5b2 0x714d637 0x714d739 0x714d831 0x714da3e 0x714dc24 0x714df73 0x714e317 0x714e547 0x714e7b1 0x714ea7e 0x714ec9c 0x714ed54 0x714ee1a 0x714eefb 0x5c46eec 0x5c46ff6 0x5c470b2 0x5c4716e 0x5c4722a 0x5c47377 0x5c47502 0x5c478c3 0x5c616c6 0xbb78627 0xbb904a5 0xbc3dd4f 0xbc3f39d 0xb90dc6d 0xb90e4a5 0x3804428 0x2958f 0x2963f 0x35eb9a4 2024-09-21T21:19:47.441 INFO:teuthology.orchestra.run.smithi040.stderr:kernel callstack: 2024-09-21T21:19:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:Suppressions used: 2024-09-21T21:19:52.096 INFO:teuthology.orchestra.run.smithi040.stderr: count bytes template 2024-09-21T21:19:52.096 INFO:teuthology.orchestra.run.smithi040.stderr: 1 8 MallocExtension::Register 2024-09-21T21:19:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:----------------------------------------------------- 2024-09-21T21:19:52.097 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:19:52.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T21:19:52.145 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T21:19:52.146 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:52.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T21:19:52.211 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:52.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T21:19:52.277 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:52.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T21:19:52.344 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:52.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T21:19:52.409 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:52.409 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T21:19:52.465 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T21:19:52.465 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBkOO9mw6PZDBAAU5Dtg5OsYNHUbE99dtuJnA== 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBnOO9mbZcGMxAANeUfoe7LFTHdzs74ccFMaw== 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBtOO9msHyAKBAAyU9TmHWP8z3uAGXBbj4vzg== 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBzOO9mLXPXHxAApR951CdJy3WsEalw3vrVhQ== 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-09-21T21:19:52.529 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBkOO9mWEMXEBAAf0tp7ZVPqQcZzpUztbz+6g== 2024-09-21T21:19:52.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T21:19:52.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:19:52.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:19:52.757 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:19:52.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T21:19:52.885 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T21:19:52.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T21:19:52.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T21:19:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.023+0000 7f4ce5232c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.023+0000 7f4ce5232c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T21:19:53.064 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:19:53.120 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T21:19:53.120 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T21:19:53.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T21:19:53.162 INFO:tasks.ceph.mon.a:Started 2024-09-21T21:19:53.163 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T21:19:53.163 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T21:19:53.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T21:19:53.165 INFO:tasks.ceph.mgr.x:Started 2024-09-21T21:19:53.165 DEBUG:tasks.ceph:set 0 configs 2024-09-21T21:19:53.165 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T21:19:53.165 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T21:19:53.250 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:19:53.247+0000 7f1b326cfc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.250 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:19:53.247+0000 7f1b326cfc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.250 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:19:53.248+0000 7f1b326cfc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.251 INFO:tasks.ceph.mgr.x.smithi040.stderr:2024-09-21T21:19:53.249+0000 7f6dc3fcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.272+0000 7f7ad8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.280+0000 7f7ad8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.353 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-09-21T21:19:53.364 INFO:tasks.ceph:check_enable_crimson: True 2024-09-21T21:19:53.364 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-21T21:19:53.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-21T21:19:53.455 INFO:tasks.ceph.mgr.x.smithi040.stderr:2024-09-21T21:19:53.453+0000 7f6dc3fcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.455 INFO:tasks.ceph.mgr.x.smithi040.stderr:2024-09-21T21:19:53.453+0000 7f6dc3fcc000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.468+0000 7f3d9fb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:53.475+0000 7f3d9fb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:53.559 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:19:53.557+0000 7f1b293f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-21T21:19:53.560 INFO:tasks.ceph.mon.a.smithi040.stderr:Please be aware that this feature is experimental, untested, 2024-09-21T21:19:53.560 INFO:tasks.ceph.mon.a.smithi040.stderr:unsupported, and may result in data corruption, data loss, 2024-09-21T21:19:53.560 INFO:tasks.ceph.mon.a.smithi040.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-21T21:19:53.560 INFO:tasks.ceph.mon.a.smithi040.stderr:feature with important data. 2024-09-21T21:19:53.560 INFO:tasks.ceph.mon.a.smithi040.stderr: 2024-09-21T21:19:54.365 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T21:19:54.365 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:54.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T21:19:54.392 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:54.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T21:19:54.457 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:19:54.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T21:19:54.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd new b457011e-431b-4c63-979a-ef60851d3722 0 2024-09-21T21:19:54.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:54.669+0000 7fc9a6d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:54.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:54.677+0000 7fc9a6d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:54.759 INFO:teuthology.orchestra.run.smithi040.stdout:0 2024-09-21T21:19:54.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd new c228c0a9-e8e6-4cdf-89f3-fde79f442e4a 1 2024-09-21T21:19:54.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:54.875+0000 7fd385e23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:54.883+0000 7fd385e23640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:54.963 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-09-21T21:19:54.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd new 42117a6b-64db-4544-846b-bc9cacf7c94e 2 2024-09-21T21:19:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:55.078+0000 7fe558111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:55.087+0000 7fe558111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:55.166 INFO:teuthology.orchestra.run.smithi040.stdout:2 2024-09-21T21:19:55.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-21T21:19:55.207 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T21:19:55.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-21T21:19:55.250 INFO:tasks.ceph.osd.0:Started 2024-09-21T21:19:55.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-21T21:19:55.281 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T21:19:55.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-21T21:19:55.323 INFO:tasks.ceph.osd.1:Started 2024-09-21T21:19:55.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-21T21:19:55.355 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T21:19:55.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-21T21:19:55.397 INFO:tasks.ceph.osd.2:Started 2024-09-21T21:19:55.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:19:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:55.566+0000 7fa10e391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:19:55.573+0000 7fa10e391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:19:55.644 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"fsid":"9e03ce4a-d956-465f-ae25-1c458f0b4938","created":"2024-09-21T21:19:53.274240+0000","modified":"2024-09-21T21:19:55.165366+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T21:19:55.165366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b457011e-431b-4c63-979a-ef60851d3722","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c228c0a9-e8e6-4cdf-89f3-fde79f442e4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"42117a6b-64db-4544-846b-bc9cacf7c94e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:19:55.659 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T21:19:55.659 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T21:19:55.960 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T21:19:56.067 INFO:teuthology.misc.health.smithi040.stderr:2024-09-21T21:19:56.061+0000 7f65e7b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:56.069 INFO:teuthology.misc.health.smithi040.stderr:2024-09-21T21:19:56.067+0000 7f65e7b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:19:56.205 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-21T21:19:56.205 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":6,"fsid":"9e03ce4a-d956-465f-ae25-1c458f0b4938","created":"2024-09-21T21:19:53.274240+0000","modified":"2024-09-21T21:19:55.165366+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T21:19:55.165366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b457011e-431b-4c63-979a-ef60851d3722","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c228c0a9-e8e6-4cdf-89f3-fde79f442e4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"42117a6b-64db-4544-846b-bc9cacf7c94e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:19:56.221 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-21T21:19:59.213 INFO:tasks.ceph.mgr.x.smithi040.stderr:2024-09-21T21:19:59.211+0000 7f6d612f6640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-21T21:20:02.524 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T21:20:02.641 INFO:teuthology.misc.health.smithi040.stderr:2024-09-21T21:20:02.636+0000 7fe2da8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:02.644 INFO:teuthology.misc.health.smithi040.stderr:2024-09-21T21:20:02.642+0000 7fe2da8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:02.844 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-21T21:20:02.845 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":9,"fsid":"9e03ce4a-d956-465f-ae25-1c458f0b4938","created":"2024-09-21T21:19:53.274240+0000","modified":"2024-09-21T21:20:02.272756+0000","last_up_change":"2024-09-21T21:20:01.212179+0000","last_in_change":"2024-09-21T21:19:55.165366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:20:02.222540+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"9","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b457011e-431b-4c63-979a-ef60851d3722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6807","nonce":3053730145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3053730145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3053730145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3053730145}]},"public_addr":"172.21.15.40:6807/3053730145","cluster_addr":"172.21.15.40:6808/3053730145","heartbeat_back_addr":"172.21.15.40:6812/3053730145","heartbeat_front_addr":"172.21.15.40:6811/3053730145","state":["exists","up"]},{"osd":1,"uuid":"c228c0a9-e8e6-4cdf-89f3-fde79f442e4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6803","nonce":899703192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":899703192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":899703192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":899703192}]},"public_addr":"172.21.15.40:6803/899703192","cluster_addr":"172.21.15.40:6804/899703192","heartbeat_back_addr":"172.21.15.40:6816/899703192","heartbeat_front_addr":"172.21.15.40:6815/899703192","state":["exists","up"]},{"osd":2,"uuid":"42117a6b-64db-4544-846b-bc9cacf7c94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4068745699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":4068745699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4068745699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6809","nonce":4068745699}]},"public_addr":"172.21.15.40:6802/4068745699","cluster_addr":"172.21.15.40:6805/4068745699","heartbeat_back_addr":"172.21.15.40:6806/4068745699","heartbeat_front_addr":"172.21.15.40:6809/4068745699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:20:02.856 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-21T21:20:02.856 INFO:tasks.ceph:Creating RBD pool 2024-09-21T21:20:02.856 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T21:20:02.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:02.962+0000 7fd307a9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:02.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:02.971+0000 7fd307a9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:03.277 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2024-09-21T21:20:03.295 DEBUG:teuthology.orchestra.run.smithi040:> rbd --cluster ceph pool init rbd 2024-09-21T21:20:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:03.398+0000 7fbcd4414ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:03.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:03.398+0000 7fbcd4414ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:03.400+0000 7fbcd4414ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:06.340 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T21:20:06.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T21:20:06.341 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T21:20:06.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:06.455+0000 7fa678c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:06.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:06.462+0000 7fa678c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:06.663 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:06.676 INFO:teuthology.orchestra.run.smithi040.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2024-09-21T21:20:06.676 INFO:tasks.ceph_manager:config epoch is 1 2024-09-21T21:20:06.677 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T21:20:06.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T21:20:06.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T21:20:06.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:06.786+0000 7f8aedf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:06.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:06.792+0000 7f8aedf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:07.053 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":3480559494},{"type":"v1","addr":"172.21.15.40:6801","nonce":3480559494}]},"active_addr":"172.21.15.40:6801/3480559494","active_change":"2024-09-21T21:19:58.204199+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2196559729}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3928807852}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2105861350}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":125622682}]}]} 2024-09-21T21:20:07.055 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T21:20:07.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T21:20:07.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:20:07.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.165+0000 7f6f1c8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.170+0000 7f6f1c8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.370 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:07.370 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"fsid":"9e03ce4a-d956-465f-ae25-1c458f0b4938","created":"2024-09-21T21:19:53.274240+0000","modified":"2024-09-21T21:20:06.296563+0000","last_up_change":"2024-09-21T21:20:01.212179+0000","last_in_change":"2024-09-21T21:19:55.165366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:20:02.222540+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:20:03.173194+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b457011e-431b-4c63-979a-ef60851d3722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6807","nonce":3053730145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3053730145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3053730145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3053730145}]},"public_addr":"172.21.15.40:6807/3053730145","cluster_addr":"172.21.15.40:6808/3053730145","heartbeat_back_addr":"172.21.15.40:6812/3053730145","heartbeat_front_addr":"172.21.15.40:6811/3053730145","state":["exists","up"]},{"osd":1,"uuid":"c228c0a9-e8e6-4cdf-89f3-fde79f442e4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6803","nonce":899703192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":899703192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":899703192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":899703192}]},"public_addr":"172.21.15.40:6803/899703192","cluster_addr":"172.21.15.40:6804/899703192","heartbeat_back_addr":"172.21.15.40:6816/899703192","heartbeat_front_addr":"172.21.15.40:6815/899703192","state":["exists","up"]},{"osd":2,"uuid":"42117a6b-64db-4544-846b-bc9cacf7c94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4068745699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":4068745699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4068745699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6809","nonce":4068745699}]},"public_addr":"172.21.15.40:6802/4068745699","cluster_addr":"172.21.15.40:6805/4068745699","heartbeat_back_addr":"172.21.15.40:6806/4068745699","heartbeat_front_addr":"172.21.15.40:6809/4068745699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:20:07.383 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T21:20:07.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:20:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.493+0000 7fcbd8751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.499+0000 7fcbd8751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:07.697 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":13,"fsid":"9e03ce4a-d956-465f-ae25-1c458f0b4938","created":"2024-09-21T21:19:53.274240+0000","modified":"2024-09-21T21:20:06.296563+0000","last_up_change":"2024-09-21T21:20:01.212179+0000","last_in_change":"2024-09-21T21:19:55.165366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:20:02.222540+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:20:03.173194+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":13,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b457011e-431b-4c63-979a-ef60851d3722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6807","nonce":3053730145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3053730145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3053730145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6811","nonce":3053730145}]},"public_addr":"172.21.15.40:6807/3053730145","cluster_addr":"172.21.15.40:6808/3053730145","heartbeat_back_addr":"172.21.15.40:6812/3053730145","heartbeat_front_addr":"172.21.15.40:6811/3053730145","state":["exists","up"]},{"osd":1,"uuid":"c228c0a9-e8e6-4cdf-89f3-fde79f442e4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6803","nonce":899703192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":899703192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":899703192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6815","nonce":899703192}]},"public_addr":"172.21.15.40:6803/899703192","cluster_addr":"172.21.15.40:6804/899703192","heartbeat_back_addr":"172.21.15.40:6816/899703192","heartbeat_front_addr":"172.21.15.40:6815/899703192","state":["exists","up"]},{"osd":2,"uuid":"42117a6b-64db-4544-846b-bc9cacf7c94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4068745699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6805","nonce":4068745699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4068745699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6809","nonce":4068745699}]},"public_addr":"172.21.15.40:6802/4068745699","cluster_addr":"172.21.15.40:6805/4068745699","heartbeat_back_addr":"172.21.15.40:6806/4068745699","heartbeat_front_addr":"172.21.15.40:6809/4068745699","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:20:07.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T21:20:07.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T21:20:07.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T21:20:07.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.827+0000 7f0741591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.827+0000 7fd7e82a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.827+0000 7f9d95591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.834+0000 7f0741591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.834+0000 7fd7e82a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:07.835+0000 7f9d95591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:07.870 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:07.870 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T21:20:07.871 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:07.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:20:07.872 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:07.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T21:20:08.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.026+0000 7fc697488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.026+0000 7fb0a8f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.033 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.026+0000 7f1edc151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.033+0000 7fb0a8f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.033+0000 7f1edc151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:08.034+0000 7fc697488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:08.245 INFO:teuthology.orchestra.run.smithi040.stdout:34359738372 2024-09-21T21:20:08.246 INFO:teuthology.orchestra.run.smithi040.stdout:34359738373 2024-09-21T21:20:08.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738372 for osd.0 2024-09-21T21:20:08.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.2 2024-09-21T21:20:08.265 INFO:teuthology.orchestra.run.smithi040.stdout:34359738373 2024-09-21T21:20:08.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.1 2024-09-21T21:20:09.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:20:09.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T21:20:09.279 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T21:20:09.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.387+0000 7f4e3b548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.397+0000 7f4e3b548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.427+0000 7fd377aa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.428+0000 7ff65fcbd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.435+0000 7fd377aa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.435+0000 7ff65fcbd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.608 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:09.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.0 2024-09-21T21:20:09.626 DEBUG:teuthology.parallel:result is None 2024-09-21T21:20:09.656 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:09.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.2 2024-09-21T21:20:09.673 DEBUG:teuthology.parallel:result is None 2024-09-21T21:20:09.676 INFO:teuthology.orchestra.run.smithi040.stdout:34359738374 2024-09-21T21:20:09.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.1 2024-09-21T21:20:09.690 DEBUG:teuthology.parallel:result is None 2024-09-21T21:20:09.690 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T21:20:09.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:20:09.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.839+0000 7f414ec7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:09.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:09.845+0000 7f414ec7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:10.043 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:10.043 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-21T21:20:10.055 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-21T21:20:08.208540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":283115520,"kb_used":80980,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":80448,"kb_avail":283034540,"statfs":{"total":289910292480,"available":289827368960,"internally_reserved":0,"allocated":462848,"data_stored":1262998,"data_compressed":60078,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":82378752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.915151"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316778+0000","last_change":"2024-09-21T21:20:06.320841+0000","last_active":"2024-09-21T21:20:06.316778+0000","last_peered":"2024-09-21T21:20:06.316778+0000","last_clean":"2024-09-21T21:20:06.316778+0000","last_became_active":"2024-09-21T21:20:04.392854+0000","last_became_peered":"2024-09-21T21:20:04.392854+0000","last_unstale":"2024-09-21T21:20:06.316778+0000","last_undegraded":"2024-09-21T21:20:06.316778+0000","last_fullsized":"2024-09-21T21:20:06.316778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.318588+0000","last_change":"2024-09-21T21:20:06.321138+0000","last_active":"2024-09-21T21:20:06.318588+0000","last_peered":"2024-09-21T21:20:06.318588+0000","last_clean":"2024-09-21T21:20:06.318588+0000","last_became_active":"2024-09-21T21:20:04.380437+0000","last_became_peered":"2024-09-21T21:20:04.380437+0000","last_unstale":"2024-09-21T21:20:06.318588+0000","last_undegraded":"2024-09-21T21:20:06.318588+0000","last_fullsized":"2024-09-21T21:20:06.318588+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.314392+0000","last_change":"2024-09-21T21:20:06.317004+0000","last_active":"2024-09-21T21:20:06.314392+0000","last_peered":"2024-09-21T21:20:06.314392+0000","last_clean":"2024-09-21T21:20:06.314392+0000","last_became_active":"2024-09-21T21:20:04.354588+0000","last_became_peered":"2024-09-21T21:20:04.354588+0000","last_unstale":"2024-09-21T21:20:06.314392+0000","last_undegraded":"2024-09-21T21:20:06.314392+0000","last_fullsized":"2024-09-21T21:20:06.314392+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.317775+0000","last_change":"2024-09-21T21:20:06.321021+0000","last_active":"2024-09-21T21:20:06.317775+0000","last_peered":"2024-09-21T21:20:06.317775+0000","last_clean":"2024-09-21T21:20:06.317775+0000","last_became_active":"2024-09-21T21:20:04.381940+0000","last_became_peered":"2024-09-21T21:20:04.381940+0000","last_unstale":"2024-09-21T21:20:06.317775+0000","last_undegraded":"2024-09-21T21:20:06.317775+0000","last_fullsized":"2024-09-21T21:20:06.317775+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"13'2","reported_seq":17,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.325488+0000","last_change":"2024-09-21T21:20:06.325488+0000","last_active":"2024-09-21T21:20:06.325488+0000","last_peered":"2024-09-21T21:20:06.325488+0000","last_clean":"2024-09-21T21:20:06.325488+0000","last_became_active":"2024-09-21T21:20:04.380437+0000","last_became_peered":"2024-09-21T21:20:04.380437+0000","last_unstale":"2024-09-21T21:20:06.325488+0000","last_undegraded":"2024-09-21T21:20:06.325488+0000","last_fullsized":"2024-09-21T21:20:06.325488+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316534+0000","last_change":"2024-09-21T21:20:06.318002+0000","last_active":"2024-09-21T21:20:06.316534+0000","last_peered":"2024-09-21T21:20:06.316534+0000","last_clean":"2024-09-21T21:20:06.316534+0000","last_became_active":"2024-09-21T21:20:04.338346+0000","last_became_peered":"2024-09-21T21:20:04.338346+0000","last_unstale":"2024-09-21T21:20:06.316534+0000","last_undegraded":"2024-09-21T21:20:06.316534+0000","last_fullsized":"2024-09-21T21:20:06.316534+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.314392+0000","last_change":"2024-09-21T21:20:06.316647+0000","last_active":"2024-09-21T21:20:06.314392+0000","last_peered":"2024-09-21T21:20:06.314392+0000","last_clean":"2024-09-21T21:20:06.314392+0000","last_became_active":"2024-09-21T21:20:04.335305+0000","last_became_peered":"2024-09-21T21:20:04.335305+0000","last_unstale":"2024-09-21T21:20:06.314392+0000","last_undegraded":"2024-09-21T21:20:06.314392+0000","last_fullsized":"2024-09-21T21:20:06.314392+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"11'1","reported_seq":16,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316989+0000","last_change":"2024-09-21T21:20:06.318944+0000","last_active":"2024-09-21T21:20:06.316989+0000","last_peered":"2024-09-21T21:20:06.316989+0000","last_clean":"2024-09-21T21:20:06.316989+0000","last_became_active":"2024-09-21T21:20:04.334077+0000","last_became_peered":"2024-09-21T21:20:04.334077+0000","last_unstale":"2024-09-21T21:20:06.316989+0000","last_undegraded":"2024-09-21T21:20:06.316989+0000","last_fullsized":"2024-09-21T21:20:06.316989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":84,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.315830+0000","last_change":"2024-09-21T21:20:03.343214+0000","last_active":"2024-09-21T21:20:06.315830+0000","last_peered":"2024-09-21T21:20:06.315830+0000","last_clean":"2024-09-21T21:20:06.315830+0000","last_became_active":"2024-09-21T21:20:03.334854+0000","last_became_peered":"2024-09-21T21:20:03.334854+0000","last_unstale":"2024-09-21T21:20:06.315830+0000","last_undegraded":"2024-09-21T21:20:06.315830+0000","last_fullsized":"2024-09-21T21:20:06.315830+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:02.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:02.272756+0000","last_clean_scrub_stamp":"2024-09-21T21:20:02.272756+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26940,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344900,"statfs":{"total":96636764160,"available":96609177600,"internally_reserved":0,"allocated":110592,"data_stored":27408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":176128,"data_stored":617795,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":176128,"data_stored":617795,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:20:10.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:20:10.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:10.164+0000 7f505e6fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:10.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:10.169+0000 7f505e6fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:10.366 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:10.367 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-21T21:20:10.380 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-21T21:20:10.208856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":283115520,"kb_used":80980,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":80448,"kb_avail":283034540,"statfs":{"total":289910292480,"available":289827368960,"internally_reserved":0,"allocated":462848,"data_stored":1262998,"data_compressed":60078,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":82378752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.915466"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316778+0000","last_change":"2024-09-21T21:20:06.320841+0000","last_active":"2024-09-21T21:20:06.316778+0000","last_peered":"2024-09-21T21:20:06.316778+0000","last_clean":"2024-09-21T21:20:06.316778+0000","last_became_active":"2024-09-21T21:20:04.392854+0000","last_became_peered":"2024-09-21T21:20:04.392854+0000","last_unstale":"2024-09-21T21:20:06.316778+0000","last_undegraded":"2024-09-21T21:20:06.316778+0000","last_fullsized":"2024-09-21T21:20:06.316778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.318588+0000","last_change":"2024-09-21T21:20:06.321138+0000","last_active":"2024-09-21T21:20:06.318588+0000","last_peered":"2024-09-21T21:20:06.318588+0000","last_clean":"2024-09-21T21:20:06.318588+0000","last_became_active":"2024-09-21T21:20:04.380437+0000","last_became_peered":"2024-09-21T21:20:04.380437+0000","last_unstale":"2024-09-21T21:20:06.318588+0000","last_undegraded":"2024-09-21T21:20:06.318588+0000","last_fullsized":"2024-09-21T21:20:06.318588+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.314392+0000","last_change":"2024-09-21T21:20:06.317004+0000","last_active":"2024-09-21T21:20:06.314392+0000","last_peered":"2024-09-21T21:20:06.314392+0000","last_clean":"2024-09-21T21:20:06.314392+0000","last_became_active":"2024-09-21T21:20:04.354588+0000","last_became_peered":"2024-09-21T21:20:04.354588+0000","last_unstale":"2024-09-21T21:20:06.314392+0000","last_undegraded":"2024-09-21T21:20:06.314392+0000","last_fullsized":"2024-09-21T21:20:06.314392+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.317775+0000","last_change":"2024-09-21T21:20:06.321021+0000","last_active":"2024-09-21T21:20:06.317775+0000","last_peered":"2024-09-21T21:20:06.317775+0000","last_clean":"2024-09-21T21:20:06.317775+0000","last_became_active":"2024-09-21T21:20:04.381940+0000","last_became_peered":"2024-09-21T21:20:04.381940+0000","last_unstale":"2024-09-21T21:20:06.317775+0000","last_undegraded":"2024-09-21T21:20:06.317775+0000","last_fullsized":"2024-09-21T21:20:06.317775+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"13'2","reported_seq":17,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.325488+0000","last_change":"2024-09-21T21:20:06.325488+0000","last_active":"2024-09-21T21:20:06.325488+0000","last_peered":"2024-09-21T21:20:06.325488+0000","last_clean":"2024-09-21T21:20:06.325488+0000","last_became_active":"2024-09-21T21:20:04.380437+0000","last_became_peered":"2024-09-21T21:20:04.380437+0000","last_unstale":"2024-09-21T21:20:06.325488+0000","last_undegraded":"2024-09-21T21:20:06.325488+0000","last_fullsized":"2024-09-21T21:20:06.325488+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316534+0000","last_change":"2024-09-21T21:20:06.318002+0000","last_active":"2024-09-21T21:20:06.316534+0000","last_peered":"2024-09-21T21:20:06.316534+0000","last_clean":"2024-09-21T21:20:06.316534+0000","last_became_active":"2024-09-21T21:20:04.338346+0000","last_became_peered":"2024-09-21T21:20:04.338346+0000","last_unstale":"2024-09-21T21:20:06.316534+0000","last_undegraded":"2024-09-21T21:20:06.316534+0000","last_fullsized":"2024-09-21T21:20:06.316534+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.314392+0000","last_change":"2024-09-21T21:20:06.316647+0000","last_active":"2024-09-21T21:20:06.314392+0000","last_peered":"2024-09-21T21:20:06.314392+0000","last_clean":"2024-09-21T21:20:06.314392+0000","last_became_active":"2024-09-21T21:20:04.335305+0000","last_became_peered":"2024-09-21T21:20:04.335305+0000","last_unstale":"2024-09-21T21:20:06.314392+0000","last_undegraded":"2024-09-21T21:20:06.314392+0000","last_fullsized":"2024-09-21T21:20:06.314392+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"11'1","reported_seq":16,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.316989+0000","last_change":"2024-09-21T21:20:06.318944+0000","last_active":"2024-09-21T21:20:06.316989+0000","last_peered":"2024-09-21T21:20:06.316989+0000","last_clean":"2024-09-21T21:20:06.316989+0000","last_became_active":"2024-09-21T21:20:04.334077+0000","last_became_peered":"2024-09-21T21:20:04.334077+0000","last_unstale":"2024-09-21T21:20:06.316989+0000","last_undegraded":"2024-09-21T21:20:06.316989+0000","last_fullsized":"2024-09-21T21:20:06.316989+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:03.276136+0000","last_clean_scrub_stamp":"2024-09-21T21:20:03.276136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":84,"reported_epoch":13,"state":"active+clean","last_fresh":"2024-09-21T21:20:06.315830+0000","last_change":"2024-09-21T21:20:03.343214+0000","last_active":"2024-09-21T21:20:06.315830+0000","last_peered":"2024-09-21T21:20:06.315830+0000","last_clean":"2024-09-21T21:20:06.315830+0000","last_became_active":"2024-09-21T21:20:03.334854+0000","last_became_peered":"2024-09-21T21:20:03.334854+0000","last_unstale":"2024-09-21T21:20:06.315830+0000","last_undegraded":"2024-09-21T21:20:06.315830+0000","last_fullsized":"2024-09-21T21:20:06.315830+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:20:02.272756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:20:02.272756+0000","last_clean_scrub_stamp":"2024-09-21T21:20:02.272756+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26940,"kb_used_data":108,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344900,"statfs":{"total":96636764160,"available":96609177600,"internally_reserved":0,"allocated":110592,"data_stored":27408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":176128,"data_stored":617795,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27020,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344820,"statfs":{"total":96636764160,"available":96609095680,"internally_reserved":0,"allocated":176128,"data_stored":617795,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30039,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:20:10.383 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T21:20:10.383 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T21:20:10.383 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T21:20:10.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T21:20:10.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:10.492+0000 7f572ab11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:10.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21T21:20:10.496+0000 7f572ab11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-21T21:20:10.735 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T21:20:10.749 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T21:20:10.749 INFO:teuthology.run_tasks:Running task cbt... 2024-09-21T21:20:10.770 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@smithi040.front.sepia.ceph.com', 'osds': ['ubuntu@smithi040.front.sepia.ceph.com'], 'mons': ['ubuntu@smithi040.front.sepia.ceph.com'], 'clients': ['ubuntu@smithi040.front.sepia.ceph.com'], 'rgws': {'ubuntu@smithi040.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randread'], 'norandommap': True, 'op_size': [4096], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2], 'cmd_path': '/home/ubuntu/cephtest/fio/fio'}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2024-09-21T21:20:10.771 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2024-09-21T21:20:10.789 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-21T21:20:10.789 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2024-09-21T21:20:10.844 DEBUG:teuthology.orchestra.run.smithi040:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2024-09-21T21:20:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2024-09-21T21:20:11.212 INFO:tasks.cbt:Installing collectl 2024-09-21T21:20:11.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2024-09-21T21:20:11.307 INFO:teuthology.orchestra.run.smithi040.stderr:--2024-09-21 21:20:11-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2024-09-21T21:20:11.465 INFO:teuthology.orchestra.run.smithi040.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.37.111, 172.64.150.145, 2606:4700:4400::6812:256f, ... 2024-09-21T21:20:11.479 INFO:teuthology.orchestra.run.smithi040.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.37.111|:443... connected. 2024-09-21T21:20:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:HTTP request sent, awaiting response... 302 Found 2024-09-21T21:20:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABm7ziL5JGoLx4k_MB4K_AAKz28sgbT-6T6siIc6mzwsPZp6Yh6D42WHOoXS5lz2dd84J1IasBN-x4FSCqJ1LGF1rW6bg%3D%3D&use_mirror=netactuate&r= [following] 2024-09-21T21:20:11.717 INFO:teuthology.orchestra.run.smithi040.stderr:--2024-09-21 21:20:11-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABm7ziL5JGoLx4k_MB4K_AAKz28sgbT-6T6siIc6mzwsPZp6Yh6D42WHOoXS5lz2dd84J1IasBN-x4FSCqJ1LGF1rW6bg%3D%3D&use_mirror=netactuate&r= 2024-09-21T21:20:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 204.68.111.105 2024-09-21T21:20:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|204.68.111.105|:443... connected. 2024-09-21T21:20:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:HTTP request sent, awaiting response... 302 Found 2024-09-21T21:20:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:Location: https://netactuate.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2024-09-21T21:20:12.098 INFO:teuthology.orchestra.run.smithi040.stderr:--2024-09-21 21:20:12-- https://netactuate.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2024-09-21T21:20:12.107 INFO:teuthology.orchestra.run.smithi040.stderr:Resolving netactuate.dl.sourceforge.net (netactuate.dl.sourceforge.net)... 104.225.3.66 2024-09-21T21:20:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:Connecting to netactuate.dl.sourceforge.net (netactuate.dl.sourceforge.net)|104.225.3.66|:443... connected. 2024-09-21T21:20:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T21:20:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:Length: 620693 (606K) [application/x-gzip] 2024-09-21T21:20:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:Saving to: ‘collectl.tar.gz’ 2024-09-21T21:20:12.281 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:20:12.300 INFO:teuthology.orchestra.run.smithi040.stderr: 0K .......... .......... .......... .......... .......... 8% 2.41M 0s 2024-09-21T21:20:12.321 INFO:teuthology.orchestra.run.smithi040.stderr: 50K .......... .......... .......... .......... .......... 16% 2.39M 0s 2024-09-21T21:20:12.342 INFO:teuthology.orchestra.run.smithi040.stderr: 100K .......... .......... .......... .......... .......... 24% 2.42M 0s 2024-09-21T21:20:12.342 INFO:teuthology.orchestra.run.smithi040.stderr: 150K .......... .......... .......... .......... .......... 32% 73.7M 0s 2024-09-21T21:20:12.342 INFO:teuthology.orchestra.run.smithi040.stderr: 200K .......... .......... .......... .......... .......... 41% 142M 0s 2024-09-21T21:20:12.364 INFO:teuthology.orchestra.run.smithi040.stderr: 250K .......... .......... .......... .......... .......... 49% 2.51M 0s 2024-09-21T21:20:12.364 INFO:teuthology.orchestra.run.smithi040.stderr: 300K .......... .......... .......... .......... .......... 57% 160M 0s 2024-09-21T21:20:12.364 INFO:teuthology.orchestra.run.smithi040.stderr: 350K .......... .......... .......... .......... .......... 65% 87.3M 0s 2024-09-21T21:20:12.364 INFO:teuthology.orchestra.run.smithi040.stderr: 400K .......... .......... .......... .......... .......... 74% 126M 0s 2024-09-21T21:20:12.366 INFO:teuthology.orchestra.run.smithi040.stderr: 450K .......... .......... .......... .......... .......... 82% 17.9M 0s 2024-09-21T21:20:12.366 INFO:teuthology.orchestra.run.smithi040.stderr: 500K .......... .......... .......... .......... .......... 90% 68.0M 0s 2024-09-21T21:20:12.382 INFO:teuthology.orchestra.run.smithi040.stderr: 550K .......... .......... .......... .......... .......... 98% 3.11M 0s 2024-09-21T21:20:12.382 INFO:teuthology.orchestra.run.smithi040.stderr: 600K ...... 100% 11.4T=0.1s 2024-09-21T21:20:12.382 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:20:12.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-21 21:20:12 (5.82 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2024-09-21T21:20:12.382 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/ 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/collectl.conf 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/ 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/SlowProc.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/BuddyInfo.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/WhySummary.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Socket.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/colmux.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Tutorial.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Gexpr.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Import.html 2024-09-21T21:20:12.414 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/style.css 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/FileNaming.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/TheMath.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Environmental.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Process.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/OutputFiles.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/HiResTime.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/OutputFormats.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/slub.jpg 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Lustre.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Documentation.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Inodes.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Graphite.html 2024-09-21T21:20:12.415 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/InputFiles.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Export.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Interrupts.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Startup.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Performance.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Messages.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Logging.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Data-detail.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/CPUs.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Matrix.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Data-brief.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/SlabInfo.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Misc.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Examples.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/OpenStack.html 2024-09-21T21:20:12.416 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Plotfiles.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Data-verbose.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Data.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Exceptions.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/RunningAsAService.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Ganglia.jpg 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/OperationalModes.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Playback.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Hello.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Sockets.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Network.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Infiniband.html 2024-09-21T21:20:12.417 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Memory.html 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/Disks.html 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/docs/NfsInfo.html 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/service/ 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/service/collectl.service 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/initd/ 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/initd/collectl-suse 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/initd/collectl-debian 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/initd/collectl-generic 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/initd/collectl 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/misc.ph 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/gexpr.ph 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/statsd.ph 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/envrules.std 2024-09-21T21:20:12.418 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/hello.ph 2024-09-21T21:20:12.419 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/vmsum.ph 2024-09-21T21:20:12.419 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/proctree.ph 2024-09-21T21:20:12.419 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/README-WINDOWS 2024-09-21T21:20:12.419 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/formatit.ph 2024-09-21T21:20:12.420 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/colmux 2024-09-21T21:20:12.420 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/INSTALL 2024-09-21T21:20:12.420 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/graphite.ph 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/RELEASE-collectl 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/vmstat.ph 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/README 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/man1/ 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/man1/collectl.1 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/man1/colmux.1 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/lexpr.ph 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/client.pl 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/collectl 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/COPYING 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/vnet.ph 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/GPL 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/UNINSTALL 2024-09-21T21:20:12.421 INFO:teuthology.orchestra.run.smithi040.stdout:collectl-4.3.1/ARTISTIC 2024-09-21T21:20:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:./INSTALL: line 90: chkconfig: command not found 2024-09-21T21:20:12.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-yaml python3-lxml librbd-devel pdsh pdsh-rcmd-ssh perf 2024-09-21T21:20:13.107 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:03:09 ago on Sat 21 Sep 2024 09:17:04 PM UTC. 2024-09-21T21:20:13.203 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-pyyaml-5.4.1-6.el9.x86_64 is already installed. 2024-09-21T21:20:13.205 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-lxml-4.6.5-3.el9.x86_64 is already installed. 2024-09-21T21:20:13.256 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: librbd-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 24 k 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: perf x86_64 5.14.0-508.el9 appstream 2.2 M 2024-09-21T21:20:13.257 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: libradospp-devel x86_64 2:19.3.0-5092.g62da54b4.el9 ceph 31 k 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout:Install 5 Packages 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.4 M 2024-09-21T21:20:13.258 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 11 M 2024-09-21T21:20:13.259 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-21T21:20:13.588 INFO:teuthology.orchestra.run.smithi040.stdout:(1/5): libradospp-devel-19.3.0-5092.g62da54b4.e 206 kB/s | 31 kB 00:00 2024-09-21T21:20:13.613 INFO:teuthology.orchestra.run.smithi040.stdout:(2/5): librbd-devel-19.3.0-5092.g62da54b4.el9.x 136 kB/s | 24 kB 00:00 2024-09-21T21:20:13.663 INFO:teuthology.orchestra.run.smithi040.stdout:(3/5): pdsh-2.34-7.el9.x86_64.rpm 1.5 MB/s | 118 kB 00:00 2024-09-21T21:20:13.688 INFO:teuthology.orchestra.run.smithi040.stdout:(4/5): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 174 kB/s | 13 kB 00:00 2024-09-21T21:20:14.006 INFO:teuthology.orchestra.run.smithi040.stdout:(5/5): perf-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 2.2 MB 00:00 2024-09-21T21:20:14.008 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:20:14.009 INFO:teuthology.orchestra.run.smithi040.stdout:Total 3.2 MB/s | 2.4 MB 00:00 2024-09-21T21:20:14.039 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-21T21:20:14.061 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-21T21:20:14.061 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-21T21:20:14.202 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-21T21:20:14.204 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-21T21:20:14.573 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-21T21:20:14.650 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/5 2024-09-21T21:20:14.689 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/5 2024-09-21T21:20:14.719 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradospp-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 3/5 2024-09-21T21:20:15.069 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librbd-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 4/5 2024-09-21T21:20:15.145 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : perf-5.14.0-508.el9.x86_64 5/5 2024-09-21T21:20:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: perf-5.14.0-508.el9.x86_64 5/5 2024-09-21T21:20:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradospp-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 1/5 2024-09-21T21:20:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2/5 2024-09-21T21:20:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : perf-5.14.0-508.el9.x86_64 3/5 2024-09-21T21:20:16.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pdsh-2.34-7.el9.x86_64 4/5 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 5/5 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: libradospp-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: librbd-devel-2:19.3.0-5092.g62da54b4.el9.x86_64 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: pdsh-2.34-7.el9.x86_64 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2024-09-21T21:20:16.405 INFO:teuthology.orchestra.run.smithi040.stdout: perf-5.14.0-508.el9.x86_64 2024-09-21T21:20:16.406 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:16.406 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-21T21:20:16.795 INFO:tasks.cbt:benchmark: librbdfio 2024-09-21T21:20:16.796 DEBUG:teuthology.orchestra.run.smithi040:> git clone -b master https://github.com/axboe/fio.git /home/ubuntu/cephtest/fio 2024-09-21T21:20:16.812 INFO:teuthology.orchestra.run.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/fio'... 2024-09-21T21:20:19.080 DEBUG:teuthology.orchestra.run.smithi040:> cd /home/ubuntu/cephtest/fio && ./configure && make 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:Operating system Linux 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:CPU x86_64 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:Big endian no 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:Compiler gcc 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:Cross compile no 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-21T21:20:19.650 INFO:teuthology.orchestra.run.smithi040.stdout:Static build no 2024-09-21T21:20:19.755 INFO:teuthology.orchestra.run.smithi040.stdout:Wordsize 64 2024-09-21T21:20:19.762 INFO:teuthology.orchestra.run.smithi040.stdout:zlib no 2024-09-21T21:20:19.809 INFO:teuthology.orchestra.run.smithi040.stdout:fcntl(F_FULLFSYNC) no 2024-09-21T21:20:19.824 INFO:teuthology.orchestra.run.smithi040.stdout:Linux AIO support no 2024-09-21T21:20:19.824 INFO:teuthology.orchestra.run.smithi040.stdout:Linux AIO support rw flags no 2024-09-21T21:20:19.893 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX AIO support yes 2024-09-21T21:20:19.894 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX AIO support needs -lrt no 2024-09-21T21:20:19.923 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX AIO fsync yes 2024-09-21T21:20:19.955 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX pshared support yes 2024-09-21T21:20:20.020 INFO:teuthology.orchestra.run.smithi040.stdout:pthread_condattr_setclock() yes 2024-09-21T21:20:20.075 INFO:teuthology.orchestra.run.smithi040.stdout:pthread_sigmask() yes 2024-09-21T21:20:20.113 INFO:teuthology.orchestra.run.smithi040.stdout:pthread_getaffinity_np() yes 2024-09-21T21:20:20.123 INFO:teuthology.orchestra.run.smithi040.stdout:Solaris AIO support no 2024-09-21T21:20:20.182 INFO:teuthology.orchestra.run.smithi040.stdout:__sync_fetch_and_add yes 2024-09-21T21:20:20.228 INFO:teuthology.orchestra.run.smithi040.stdout:__sync_synchronize yes 2024-09-21T21:20:20.262 INFO:teuthology.orchestra.run.smithi040.stdout:__sync_val_compare_and_swap yes 2024-09-21T21:20:20.270 INFO:teuthology.orchestra.run.smithi040.stdout:libverbs no 2024-09-21T21:20:20.294 INFO:teuthology.orchestra.run.smithi040.stdout:rdmacm no 2024-09-21T21:20:20.326 INFO:teuthology.orchestra.run.smithi040.stdout:asprintf() yes 2024-09-21T21:20:20.360 INFO:teuthology.orchestra.run.smithi040.stdout:vasprintf() yes 2024-09-21T21:20:20.396 INFO:teuthology.orchestra.run.smithi040.stdout:Linux fallocate yes 2024-09-21T21:20:20.429 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX fadvise yes 2024-09-21T21:20:20.461 INFO:teuthology.orchestra.run.smithi040.stdout:POSIX fallocate yes 2024-09-21T21:20:20.492 INFO:teuthology.orchestra.run.smithi040.stdout:sched_setaffinity(3 arg) yes 2024-09-21T21:20:20.493 INFO:teuthology.orchestra.run.smithi040.stdout:sched_setaffinity(2 arg) no 2024-09-21T21:20:20.525 INFO:teuthology.orchestra.run.smithi040.stdout:clock_gettime yes 2024-09-21T21:20:20.559 INFO:teuthology.orchestra.run.smithi040.stdout:CLOCK_MONOTONIC yes 2024-09-21T21:20:20.592 INFO:teuthology.orchestra.run.smithi040.stdout:clockid_t yes 2024-09-21T21:20:20.633 INFO:teuthology.orchestra.run.smithi040.stdout:gettimeofday yes 2024-09-21T21:20:20.667 INFO:teuthology.orchestra.run.smithi040.stdout:fdatasync yes 2024-09-21T21:20:20.699 INFO:teuthology.orchestra.run.smithi040.stdout:pipe() yes 2024-09-21T21:20:20.730 INFO:teuthology.orchestra.run.smithi040.stdout:pipe2() yes 2024-09-21T21:20:20.763 INFO:teuthology.orchestra.run.smithi040.stdout:pread() yes 2024-09-21T21:20:20.843 INFO:teuthology.orchestra.run.smithi040.stdout:sync_file_range yes 2024-09-21T21:20:20.851 INFO:teuthology.orchestra.run.smithi040.stdout:ASharedMemory_create no 2024-09-21T21:20:20.879 INFO:teuthology.orchestra.run.smithi040.stdout:EXT4 move extent yes 2024-09-21T21:20:20.912 INFO:teuthology.orchestra.run.smithi040.stdout:Linux splice(2) yes 2024-09-21T21:20:20.921 INFO:teuthology.orchestra.run.smithi040.stdout:libnuma no 2024-09-21T21:20:20.951 INFO:teuthology.orchestra.run.smithi040.stdout:strsep yes 2024-09-21T21:20:20.982 INFO:teuthology.orchestra.run.smithi040.stdout:strcasestr yes 2024-09-21T21:20:20.997 INFO:teuthology.orchestra.run.smithi040.stdout:strlcat no 2024-09-21T21:20:21.033 INFO:teuthology.orchestra.run.smithi040.stdout:getopt_long_only() yes 2024-09-21T21:20:21.099 INFO:teuthology.orchestra.run.smithi040.stdout:inet_aton yes 2024-09-21T21:20:21.132 INFO:teuthology.orchestra.run.smithi040.stdout:socklen_t yes 2024-09-21T21:20:21.165 INFO:teuthology.orchestra.run.smithi040.stdout:__thread yes 2024-09-21T21:20:21.203 INFO:teuthology.orchestra.run.smithi040.stdout:RUSAGE_THREAD yes 2024-09-21T21:20:21.234 INFO:teuthology.orchestra.run.smithi040.stdout:SCHED_IDLE yes 2024-09-21T21:20:21.271 INFO:teuthology.orchestra.run.smithi040.stdout:TCP_NODELAY yes 2024-09-21T21:20:21.317 INFO:teuthology.orchestra.run.smithi040.stdout:vsock yes 2024-09-21T21:20:21.354 INFO:teuthology.orchestra.run.smithi040.stdout:Net engine window_size yes 2024-09-21T21:20:21.390 INFO:teuthology.orchestra.run.smithi040.stdout:TCP_MAXSEG yes 2024-09-21T21:20:21.421 INFO:teuthology.orchestra.run.smithi040.stdout:RLIMIT_MEMLOCK yes 2024-09-21T21:20:21.456 INFO:teuthology.orchestra.run.smithi040.stdout:pwritev/preadv yes 2024-09-21T21:20:21.490 INFO:teuthology.orchestra.run.smithi040.stdout:pwritev2/preadv2 yes 2024-09-21T21:20:21.530 INFO:teuthology.orchestra.run.smithi040.stdout:IPv6 helpers yes 2024-09-21T21:20:21.546 INFO:teuthology.orchestra.run.smithi040.stdout:http engine no 2024-09-21T21:20:21.697 INFO:teuthology.orchestra.run.smithi040.stdout:Rados engine yes 2024-09-21T21:20:21.891 INFO:teuthology.orchestra.run.smithi040.stdout:Rados Block Device engine yes 2024-09-21T21:20:22.081 INFO:teuthology.orchestra.run.smithi040.stdout:rbd_poll yes 2024-09-21T21:20:22.272 INFO:teuthology.orchestra.run.smithi040.stdout:rbd_invalidate_cache yes 2024-09-21T21:20:22.304 INFO:teuthology.orchestra.run.smithi040.stdout:setvbuf yes 2024-09-21T21:20:22.311 INFO:teuthology.orchestra.run.smithi040.stdout:Gluster API engine no 2024-09-21T21:20:22.328 INFO:teuthology.orchestra.run.smithi040.stdout:s390_z196_facilities no 2024-09-21T21:20:22.328 INFO:teuthology.orchestra.run.smithi040.stdout:HDFS engine no 2024-09-21T21:20:22.375 INFO:teuthology.orchestra.run.smithi040.stdout:MTD yes 2024-09-21T21:20:22.382 INFO:teuthology.orchestra.run.smithi040.stdout:libpmem no 2024-09-21T21:20:22.382 INFO:teuthology.orchestra.run.smithi040.stdout:libpmem1_5 no 2024-09-21T21:20:22.390 INFO:teuthology.orchestra.run.smithi040.stdout:libpmem2 no 2024-09-21T21:20:22.390 INFO:teuthology.orchestra.run.smithi040.stdout:PMDK dev-dax engine no 2024-09-21T21:20:22.390 INFO:teuthology.orchestra.run.smithi040.stdout:PMDK libpmem engine no 2024-09-21T21:20:22.397 INFO:teuthology.orchestra.run.smithi040.stdout:DDN's Infinite Memory Engine no 2024-09-21T21:20:22.397 INFO:teuthology.orchestra.run.smithi040.stdout:iscsi engine no 2024-09-21T21:20:22.397 INFO:teuthology.orchestra.run.smithi040.stdout:NBD engine no 2024-09-21T21:20:22.407 INFO:teuthology.orchestra.run.smithi040.stdout:DAOS File System (dfs) Engine no 2024-09-21T21:20:22.416 INFO:teuthology.orchestra.run.smithi040.stdout:NFS engine no 2024-09-21T21:20:22.416 INFO:teuthology.orchestra.run.smithi040.stdout:lex/yacc for arithmetic no 2024-09-21T21:20:22.450 INFO:teuthology.orchestra.run.smithi040.stdout:getmntent yes 2024-09-21T21:20:22.505 INFO:teuthology.orchestra.run.smithi040.stdout:getmntinfo no 2024-09-21T21:20:22.563 INFO:teuthology.orchestra.run.smithi040.stdout:Static Assert yes 2024-09-21T21:20:22.592 INFO:teuthology.orchestra.run.smithi040.stdout:bool yes 2024-09-21T21:20:22.627 INFO:teuthology.orchestra.run.smithi040.stdout:strndup yes 2024-09-21T21:20:22.635 INFO:teuthology.orchestra.run.smithi040.stdout:Valgrind headers no 2024-09-21T21:20:22.665 INFO:teuthology.orchestra.run.smithi040.stdout:Zoned block device support yes 2024-09-21T21:20:22.694 INFO:teuthology.orchestra.run.smithi040.stdout:Zoned block device capacity yes 2024-09-21T21:20:22.701 INFO:teuthology.orchestra.run.smithi040.stdout:libzbc engine no 2024-09-21T21:20:22.732 INFO:teuthology.orchestra.run.smithi040.stdout:NVMe uring command support yes 2024-09-21T21:20:22.747 INFO:teuthology.orchestra.run.smithi040.stdout:xnvme engine no 2024-09-21T21:20:22.755 INFO:teuthology.orchestra.run.smithi040.stdout:isal no 2024-09-21T21:20:22.771 INFO:teuthology.orchestra.run.smithi040.stdout:libblkio engine no 2024-09-21T21:20:22.771 INFO:teuthology.orchestra.run.smithi040.stdout:march_armv8_a_crc_crypto no 2024-09-21T21:20:22.771 INFO:teuthology.orchestra.run.smithi040.stdout:cuda no 2024-09-21T21:20:22.771 INFO:teuthology.orchestra.run.smithi040.stdout:libcufile no 2024-09-21T21:20:22.800 INFO:teuthology.orchestra.run.smithi040.stdout:Build march=native yes 2024-09-21T21:20:22.808 INFO:teuthology.orchestra.run.smithi040.stdout:CUnit no 2024-09-21T21:20:22.837 INFO:teuthology.orchestra.run.smithi040.stdout:__kernel_rwf_t yes 2024-09-21T21:20:22.866 INFO:teuthology.orchestra.run.smithi040.stdout:-Wimplicit-fallthrough=2 yes 2024-09-21T21:20:22.898 INFO:teuthology.orchestra.run.smithi040.stdout:-Wno-stringop-truncation yes 2024-09-21T21:20:22.941 INFO:teuthology.orchestra.run.smithi040.stdout:MADV_HUGEPAGE yes 2024-09-21T21:20:22.973 INFO:teuthology.orchestra.run.smithi040.stdout:gettid yes 2024-09-21T21:20:23.026 INFO:teuthology.orchestra.run.smithi040.stdout:statx(2)/libc yes 2024-09-21T21:20:23.074 INFO:teuthology.orchestra.run.smithi040.stdout:statx(2)/syscall yes 2024-09-21T21:20:23.077 INFO:teuthology.orchestra.run.smithi040.stdout:Windows PDB generation no 2024-09-21T21:20:23.108 INFO:teuthology.orchestra.run.smithi040.stdout:timerfd_create yes 2024-09-21T21:20:23.111 INFO:teuthology.orchestra.run.smithi040.stdout:Consider installing zlib1g-dev (zlib-devel) as some fio features depend on it. 2024-09-21T21:20:23.111 INFO:teuthology.orchestra.run.smithi040.stdout:Lib-based ioengines dynamic no 2024-09-21T21:20:23.177 INFO:teuthology.orchestra.run.smithi040.stdout:TCMalloc support yes 2024-09-21T21:20:23.179 INFO:teuthology.orchestra.run.smithi040.stdout:seed_buckets 4 2024-09-21T21:20:23.262 INFO:teuthology.orchestra.run.smithi040.stderr:FIO_VERSION = fio-3.37-124-gbcd4 2024-09-21T21:20:23.295 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc16.o 2024-09-21T21:20:23.388 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc32.o 2024-09-21T21:20:23.427 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc32c-arm64.o 2024-09-21T21:20:23.518 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc32c-intel.o 2024-09-21T21:20:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc32c.o 2024-09-21T21:20:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc64.o 2024-09-21T21:20:23.649 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crc7.o 2024-09-21T21:20:23.685 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/crct10dif_common.o 2024-09-21T21:20:23.722 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/fnv.o 2024-09-21T21:20:23.765 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/md5.o 2024-09-21T21:20:23.869 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/murmur3.o 2024-09-21T21:20:23.914 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/sha1.o 2024-09-21T21:20:24.105 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/sha256.o 2024-09-21T21:20:24.418 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/sha3.o 2024-09-21T21:20:24.571 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/sha512.o 2024-09-21T21:20:24.692 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/test.o 2024-09-21T21:20:24.867 INFO:teuthology.orchestra.run.smithi040.stdout: CC crc/xxhash.o 2024-09-21T21:20:24.967 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/axmap.o 2024-09-21T21:20:25.096 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/bloom.o 2024-09-21T21:20:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/flist_sort.o 2024-09-21T21:20:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/gauss.o 2024-09-21T21:20:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/getrusage.o 2024-09-21T21:20:25.414 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/hweight.o 2024-09-21T21:20:25.455 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/ieee754.o 2024-09-21T21:20:25.507 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/lfsr.o 2024-09-21T21:20:25.614 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/memalign.o 2024-09-21T21:20:25.660 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/memcpy.o 2024-09-21T21:20:25.810 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/mountcheck.o 2024-09-21T21:20:25.855 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/num2str.o 2024-09-21T21:20:25.924 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/output_buffer.o 2024-09-21T21:20:25.974 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/pattern.o 2024-09-21T21:20:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/prio_tree.o 2024-09-21T21:20:26.250 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/rand.o 2024-09-21T21:20:26.450 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/rbtree.o 2024-09-21T21:20:26.582 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/strntol.o 2024-09-21T21:20:26.632 INFO:teuthology.orchestra.run.smithi040.stdout: CC lib/zipf.o 2024-09-21T21:20:26.756 INFO:teuthology.orchestra.run.smithi040.stdout: CC gettime.o 2024-09-21T21:20:27.066 INFO:teuthology.orchestra.run.smithi040.stdout: CC ioengines.o 2024-09-21T21:20:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: CC init.o 2024-09-21T21:20:28.162 INFO:teuthology.orchestra.run.smithi040.stdout: CC stat.o 2024-09-21T21:20:30.099 INFO:teuthology.orchestra.run.smithi040.stdout: CC log.o 2024-09-21T21:20:30.239 INFO:teuthology.orchestra.run.smithi040.stdout: CC time.o 2024-09-21T21:20:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: CC filesetup.o 2024-09-21T21:20:31.126 INFO:teuthology.orchestra.run.smithi040.stdout: CC eta.o 2024-09-21T21:20:31.468 INFO:teuthology.orchestra.run.smithi040.stdout: CC verify.o 2024-09-21T21:20:32.178 INFO:teuthology.orchestra.run.smithi040.stdout: CC memory.o 2024-09-21T21:20:32.358 INFO:teuthology.orchestra.run.smithi040.stdout: CC io_u.o 2024-09-21T21:20:33.181 INFO:teuthology.orchestra.run.smithi040.stdout: CC parse.o 2024-09-21T21:20:33.657 INFO:teuthology.orchestra.run.smithi040.stdout: CC fio_sem.o 2024-09-21T21:20:33.769 INFO:teuthology.orchestra.run.smithi040.stdout: CC rwlock.o 2024-09-21T21:20:33.859 INFO:teuthology.orchestra.run.smithi040.stdout: CC pshared.o 2024-09-21T21:20:33.928 INFO:teuthology.orchestra.run.smithi040.stdout: CC options.o 2024-09-21T21:20:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: CC smalloc.o 2024-09-21T21:20:34.977 INFO:teuthology.orchestra.run.smithi040.stdout: CC filehash.o 2024-09-21T21:20:35.136 INFO:teuthology.orchestra.run.smithi040.stdout: CC profile.o 2024-09-21T21:20:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: CC debug.o 2024-09-21T21:20:35.306 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/cpu.o 2024-09-21T21:20:35.452 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/mmap.o 2024-09-21T21:20:35.630 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/sync.o 2024-09-21T21:20:35.871 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/null.o 2024-09-21T21:20:35.992 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/net.o 2024-09-21T21:20:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/ftruncate.o 2024-09-21T21:20:36.552 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/fileoperations.o 2024-09-21T21:20:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/exec.o 2024-09-21T21:20:36.880 INFO:teuthology.orchestra.run.smithi040.stdout: CC server.o 2024-09-21T21:20:37.665 INFO:teuthology.orchestra.run.smithi040.stdout: CC client.o 2024-09-21T21:20:38.234 INFO:teuthology.orchestra.run.smithi040.stdout: CC iolog.o 2024-09-21T21:20:38.710 INFO:teuthology.orchestra.run.smithi040.stdout: CC backend.o 2024-09-21T21:20:39.603 INFO:teuthology.orchestra.run.smithi040.stdout: CC libfio.o 2024-09-21T21:20:39.789 INFO:teuthology.orchestra.run.smithi040.stdout: CC flow.o 2024-09-21T21:20:39.914 INFO:teuthology.orchestra.run.smithi040.stdout: CC cconv.o 2024-09-21T21:20:40.290 INFO:teuthology.orchestra.run.smithi040.stdout: CC gettime-thread.o 2024-09-21T21:20:40.402 INFO:teuthology.orchestra.run.smithi040.stdout: CC helpers.o 2024-09-21T21:20:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: CC json.o 2024-09-21T21:20:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: CC idletime.o 2024-09-21T21:20:40.894 INFO:teuthology.orchestra.run.smithi040.stdout: CC td_error.o 2024-09-21T21:20:40.997 INFO:teuthology.orchestra.run.smithi040.stdout: CC profiles/tiobench.o 2024-09-21T21:20:41.097 INFO:teuthology.orchestra.run.smithi040.stdout: CC profiles/act.o 2024-09-21T21:20:41.268 INFO:teuthology.orchestra.run.smithi040.stdout: CC io_u_queue.o 2024-09-21T21:20:41.321 INFO:teuthology.orchestra.run.smithi040.stdout: CC filelock.o 2024-09-21T21:20:41.453 INFO:teuthology.orchestra.run.smithi040.stdout: CC workqueue.o 2024-09-21T21:20:41.642 INFO:teuthology.orchestra.run.smithi040.stdout: CC rate-submit.o 2024-09-21T21:20:41.798 INFO:teuthology.orchestra.run.smithi040.stdout: CC optgroup.o 2024-09-21T21:20:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: CC helper_thread.o 2024-09-21T21:20:42.040 INFO:teuthology.orchestra.run.smithi040.stdout: CC steadystate.o 2024-09-21T21:20:42.294 INFO:teuthology.orchestra.run.smithi040.stdout: CC zone-dist.o 2024-09-21T21:20:42.436 INFO:teuthology.orchestra.run.smithi040.stdout: CC zbd.o 2024-09-21T21:20:43.165 INFO:teuthology.orchestra.run.smithi040.stdout: CC dedupe.o 2024-09-21T21:20:43.315 INFO:teuthology.orchestra.run.smithi040.stdout: CC dataplacement.o 2024-09-21T21:20:43.497 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/posixaio.o 2024-09-21T21:20:43.642 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/falloc.o 2024-09-21T21:20:43.764 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/e4defrag.o 2024-09-21T21:20:43.906 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/splice.o 2024-09-21T21:20:44.105 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/asprintf.o 2024-09-21T21:20:44.144 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/strlcat.o 2024-09-21T21:20:44.191 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/mtd.o 2024-09-21T21:20:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/libmtd.o 2024-09-21T21:20:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/libmtd_legacy.o 2024-09-21T21:20:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: CC diskutil.o 2024-09-21T21:20:45.142 INFO:teuthology.orchestra.run.smithi040.stdout: CC fifo.o 2024-09-21T21:20:45.201 INFO:teuthology.orchestra.run.smithi040.stdout: CC blktrace.o 2024-09-21T21:20:45.549 INFO:teuthology.orchestra.run.smithi040.stdout: CC cgroup.o 2024-09-21T21:20:45.705 INFO:teuthology.orchestra.run.smithi040.stdout: CC trim.o 2024-09-21T21:20:45.829 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/sg.o 2024-09-21T21:20:46.229 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/linux-dev-lookup.o 2024-09-21T21:20:46.292 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/io_uring.o 2024-09-21T21:20:46.711 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/nvme.o 2024-09-21T21:20:47.036 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/linux-blkzoned.o 2024-09-21T21:20:47.201 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/cmdprio.o 2024-09-21T21:20:47.450 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/rados.o 2024-09-21T21:20:47.632 INFO:teuthology.orchestra.run.smithi040.stdout: CC engines/rbd.o 2024-09-21T21:20:47.829 INFO:teuthology.orchestra.run.smithi040.stdout: CC fio.o 2024-09-21T21:20:47.926 INFO:teuthology.orchestra.run.smithi040.stdout: LINK fio 2024-09-21T21:20:48.220 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/log.o 2024-09-21T21:20:48.266 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/genzipf.o 2024-09-21T21:20:48.415 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/strcasestr.o 2024-09-21T21:20:48.442 INFO:teuthology.orchestra.run.smithi040.stdout: CC oslib/strndup.o 2024-09-21T21:20:48.467 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/fio-genzipf 2024-09-21T21:20:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/btrace2fio.o 2024-09-21T21:20:49.083 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/fio-btrace2fio 2024-09-21T21:20:49.264 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/verify-state.o 2024-09-21T21:20:49.367 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/debug.o 2024-09-21T21:20:49.404 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/fio-verify-state 2024-09-21T21:20:49.589 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/stest.o 2024-09-21T21:20:49.669 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/arch.o 2024-09-21T21:20:49.701 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/stest 2024-09-21T21:20:49.885 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/ieee754.o 2024-09-21T21:20:49.928 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/ieee754 2024-09-21T21:20:50.106 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/axmap.o 2024-09-21T21:20:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/axmap 2024-09-21T21:20:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/lfsr-test.o 2024-09-21T21:20:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/lfsr-test 2024-09-21T21:20:50.672 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/gen-rand.o 2024-09-21T21:20:50.773 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/gen-rand 2024-09-21T21:20:50.950 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/memlock.o 2024-09-21T21:20:51.013 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/memlock 2024-09-21T21:20:51.191 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/read-to-pipe-async.o 2024-09-21T21:20:51.392 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/read-to-pipe-async 2024-09-21T21:20:51.572 INFO:teuthology.orchestra.run.smithi040.stdout: CC t/io_uring.o 2024-09-21T21:20:52.134 INFO:teuthology.orchestra.run.smithi040.stdout: LINK t/io_uring 2024-09-21T21:20:52.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2024-09-21T21:20:52.341 DEBUG:teuthology.orchestra.run.smithi040:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2024-09-21T21:20:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - Settings.cluster: 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'clients': ['ubuntu@smithi040.front.sepia.ceph.com'], 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'head': 'ubuntu@smithi040.front.sepia.ceph.com', 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'is_teuthology': True, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'iterations': 1, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'mons': ['ubuntu@smithi040.front.sepia.ceph.com'], 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'osds': ['ubuntu@smithi040.front.sepia.ceph.com'], 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'osds_per_node': 3, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'pid_dir': '/var/run/ceph', 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'rebuild_every_test': False, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'rgws': {'ubuntu@smithi040.front.sepia.ceph.com': None}, 2024-09-21T21:20:52.584 INFO:teuthology.orchestra.run.smithi040.stderr: 'tmp_dir': '/tmp/cbt', 2024-09-21T21:20:52.585 INFO:teuthology.orchestra.run.smithi040.stderr: 'use_existing': True, 2024-09-21T21:20:52.585 INFO:teuthology.orchestra.run.smithi040.stderr: 'user': 'ubuntu'} 2024-09-21T21:20:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-c9606d7c 2024-09-21T21:20:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=if [ -d "/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread" ]; then rm -rf /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread; fi 2024-09-21T21:20:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread 2024-09-21T21:20:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - INFO - cbt - Pausing for 60s for idle monitoring. 2024-09-21T21:20:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/idle_monitoring/collectl 2024-09-21T21:20:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2024-09-21T21:20:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/idle_monitoring/perf 2024-09-21T21:20:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - PerfMonitoring: in local_node 2024-09-21T21:20:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:21:20:52 - DEBUG - cbt - pid_dir: /var/run/ceph 2024-09-21T21:21:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=pkill -SIGINT -f collectl 2024-09-21T21:21:52.695 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo pkill -SIGINT -f perf\ 2024-09-21T21:21:52.739 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/* 2024-09-21T21:21:52.755 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=for i in /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/* ; do cp -r ${i} /home/ubuntu/cephtest/archive/cbt/results/00000000/id-c9606d7c/$(basename ${i}).smithi040.front.sepia.ceph.com ; done 2024-09-21T21:21:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - INFO - cbt - Creating fio images... 2024-09-21T21:21:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/pool_monitoring/collectl 2024-09-21T21:21:52.762 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2024-09-21T21:21:52.764 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randread/pool_monitoring/perf 2024-09-21T21:21:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - PerfMonitoring: in local_node 2024-09-21T21:21:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - pid_dir: /var/run/ceph 2024-09-21T21:21:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:52 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete cbt-librbdfio cbt-librbdfio --yes-i-really-really-mean-it 2024-09-21T21:21:53.085 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:53 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create cbt-librbdfio 128 128 2024-09-21T21:21:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:54 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable cbt-librbdfio rbd 2024-09-21T21:21:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:55 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio size 3 2024-09-21T21:21:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:56 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio min_size 2 2024-09-21T21:21:57.428 INFO:teuthology.orchestra.run.smithi040.stderr:21:21:57 - INFO - cbt - Checking Health after pool creation. 2024-09-21T21:22:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:02 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=/usr/bin/ceph -c /etc/ceph/ceph.conf health 2024-09-21T21:22:02.786 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:02 - INFO - cbt - Final Pool Health Check. 2024-09-21T21:22:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:07 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=/usr/bin/ceph -c /etc/ceph/ceph.conf health 2024-09-21T21:22:08.130 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:08 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=hostname -f 2024-09-21T21:22:08.132 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:08 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=/usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-smithi040.front.sepia.ceph.com-0 --size 4096 --pool cbt-librbdfio --order 22 2024-09-21T21:22:09.622 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=True args=/usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-smithi040.front.sepia.ceph.com-1 --size 4096 --pool cbt-librbdfio --order 22 2024-09-21T21:22:09.738 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=pkill -SIGINT -f collectl 2024-09-21T21:22:09.763 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo pkill -SIGINT -f perf\ 2024-09-21T21:22:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - INFO - cbt - Attempting to prefill fio images... 2024-09-21T21:22:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2024-09-21T21:22:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:21:22:09 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2024-09-21T21:23:35.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-21T21:23:41.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-09-21T21:23:46.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2024-09-21T21:23:52.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2024-09-21T21:23:57.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2024-09-21T21:24:02.885 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~27s 2024-09-21T21:24:08.287 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~32s 2024-09-21T21:24:13.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~38s 2024-09-21T21:24:19.093 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~43s 2024-09-21T21:24:24.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~49s 2024-09-21T21:24:29.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~54s 2024-09-21T21:24:35.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~59s 2024-09-21T21:24:40.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-09-21T21:24:46.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2024-09-21T21:24:51.482 INFO:teuthology.orchestra.run.smithi040.stderr:21:24:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2024-09-21T21:24:51.483 INFO:teuthology.orchestra.run.smithi040.stderr:21:24:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=True args=sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2024-09-21T21:24:51.512 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~76s 2024-09-21T21:24:56.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~81s 2024-09-21T21:25:02.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~86s 2024-09-21T21:25:07.722 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~92s 2024-09-21T21:25:13.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~97s 2024-09-21T21:25:18.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~103s 2024-09-21T21:25:23.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~108s 2024-09-21T21:25:29.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~113s 2024-09-21T21:25:34.737 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~119s 2024-09-21T21:25:40.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~124s 2024-09-21T21:25:45.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2024-09-21T21:25:50.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~135s 2024-09-21T21:25:56.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~140s 2024-09-21T21:26:01.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~146s 2024-09-21T21:26:07.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~151s 2024-09-21T21:26:12.557 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~157s 2024-09-21T21:26:17.960 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~162s 2024-09-21T21:26:23.364 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~167s 2024-09-21T21:26:28.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~173s 2024-09-21T21:26:34.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~178s 2024-09-21T21:26:39.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-21T21:26:39.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~184s 2024-09-21T21:26:44.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-09-21T21:26:44.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~189s 2024-09-21T21:26:50.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~11s 2024-09-21T21:26:50.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~194s 2024-09-21T21:26:55.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2024-09-21T21:26:55.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2024-09-21T21:27:00.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~21s 2024-09-21T21:27:00.684 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2024-09-21T21:27:05.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2024-09-21T21:27:05.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~210s 2024-09-21T21:27:11.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~32s 2024-09-21T21:27:11.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~216s 2024-09-21T21:27:16.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~37s 2024-09-21T21:27:16.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~221s 2024-09-21T21:27:21.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~42s 2024-09-21T21:27:21.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~226s 2024-09-21T21:27:27.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~48s 2024-09-21T21:27:27.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~231s 2024-09-21T21:27:32.499 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~53s 2024-09-21T21:27:32.500 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~237s 2024-09-21T21:27:33.304 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:33.302+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:37.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~58s 2024-09-21T21:27:37.802 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~242s 2024-09-21T21:27:38.304 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:38.303+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:43.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~64s 2024-09-21T21:27:43.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~247s 2024-09-21T21:27:43.305 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:43.304+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:48.306 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:48.304+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:48.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~69s 2024-09-21T21:27:48.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~253s 2024-09-21T21:27:53.307 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:53.304+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:53.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~74s 2024-09-21T21:27:53.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~258s 2024-09-21T21:27:58.307 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:27:58.305+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:27:59.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2024-09-21T21:27:59.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~263s 2024-09-21T21:28:03.308 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:03.306+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:04.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2024-09-21T21:28:04.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~269s 2024-09-21T21:28:08.309 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:08.307+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:09.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2024-09-21T21:28:09.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~274s 2024-09-21T21:28:13.309 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:13.308+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:14.922 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2024-09-21T21:28:14.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~279s 2024-09-21T21:28:18.310 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:18.308+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:20.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~101s 2024-09-21T21:28:20.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~284s 2024-09-21T21:28:23.311 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:23.309+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:25.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~106s 2024-09-21T21:28:25.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~290s 2024-09-21T21:28:28.311 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:28.309+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:30.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~111s 2024-09-21T21:28:30.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~295s 2024-09-21T21:28:33.312 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:33.310+0000 7f1b2bbf9640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:172.21.15.40:6807/3053730145 for 29sec e23 v23) 2024-09-21T21:28:36.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~117s 2024-09-21T21:28:36.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~300s 2024-09-21T21:28:36.134 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-09-21T21:28:36.436 INFO:tasks.ceph.osd.1:Sent signal 15 2024-09-21T21:28:36.437 INFO:tasks.ceph.mon.a:Sent signal 15 2024-09-21T21:28:36.437 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-09-21T21:28:36.438 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:36.436+0000 7f1b2ebff640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 28806) UID: 0 2024-09-21T21:28:36.438 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-21T21:28:36.436+0000 7f1b2ebff640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T21:28:36.438 INFO:tasks.ceph.mgr.x.smithi040.stderr:2024-09-21T21:28:36.436+0000 7f6dbdbf7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 28809) UID: 0 2024-09-22T05:09:57.924 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-09-22T05:09:57.946 DEBUG:teuthology.task.console_log:Killing console logger for smithi040 2024-09-22T05:09:57.946 DEBUG:teuthology.exit:Finished running handlers