2023-03-20T09:52:08.695 INFO:root:teuthology version: 0.0.1.dev59+g7f7f7c3 2023-03-20T09:52:09.458 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} --name yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642 --verbose -- /home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642/orig.config.yaml 2023-03-20T09:52:09.496 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-20T09:52:10.920 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642 branch: reef description: rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} email: null first_in_suite: false job_id: '7212642' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 setgroup: ceph setuser: ceph global: osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c wait-for-scrub: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c rgw: frontend: beast storage classes: LUKEWARM, FROZEN selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c sleep_before_teardown: 0 suite: rgw suite_branch: reef suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_a6dd66830c12498d02f3894e352dd8b49a7bab4c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHQ4yea4haxdr0Tc1sKDpUA30E26tyZ5GHPbuAYKSkdjA+Xn6I76KU6CIrexLrMe8XM/8H2FjJETZjXoFBoIke8= smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC74ZJP/jMnXnRPjq0oFFcua5P/jsurKd9v8GUcROUrbaHWZE5e2Y38QftDbrLEzt0tAvFpIUEMvvJDd/BYJbQ= tasks: - install: null - ceph: null - openssl_keys: null - rgw: client.0: null - rabbitmq: client.0: null - notification-tests: client.0: extra_attr: - amqp_test rgw_server: client.0 - kafka: client.0: kafka_version: 2.6.0 - notification-tests: client.0: extra_attr: - kafka_test rgw_server: client.0 - notification-tests: client.0: rgw_server: client.0 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 7f7f7c33a372789d0252bbbcd5798300b0f044d0 timestamp: 2023-03-17_23:46:23 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4737 2023-03-20T09:52:12.391 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_a6dd66830c12498d02f3894e352dd8b49a7bab4c/qa; will attempt to use it 2023-03-20T09:52:12.392 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_a6dd66830c12498d02f3894e352dd8b49a7bab4c/qa/tasks 2023-03-20T09:52:12.392 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-20T09:52:12.394 INFO:teuthology.task.internal:Checking packages... 2023-03-20T09:52:13.635 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a6dd66830c12498d02f3894e352dd8b49a7bab4c' 2023-03-20T09:52:14.988 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-20T09:52:14.988 INFO:teuthology.packaging:ref: None 2023-03-20T09:52:14.988 INFO:teuthology.packaging:tag: None 2023-03-20T09:52:14.988 INFO:teuthology.packaging:branch: reef 2023-03-20T09:52:14.988 INFO:teuthology.packaging:sha1: a6dd66830c12498d02f3894e352dd8b49a7bab4c 2023-03-20T09:52:14.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-03-20T09:52:15.193 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2745.g8898ee66 2023-03-20T09:52:17.017 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-20T09:52:17.023 INFO:teuthology.task.internal:no buildpackages task found 2023-03-20T09:52:18.398 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-20T09:52:18.405 INFO:teuthology.task.internal:Saving configuration 2023-03-20T09:52:18.912 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-20T09:52:19.845 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-20T09:52:21.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-20 09:45:52.151962', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHQ4yea4haxdr0Tc1sKDpUA30E26tyZ5GHPbuAYKSkdjA+Xn6I76KU6CIrexLrMe8XM/8H2FjJETZjXoFBoIke8='} 2023-03-20T09:52:22.731 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-20 09:45:52.153532', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC74ZJP/jMnXnRPjq0oFFcua5P/jsurKd9v8GUcROUrbaHWZE5e2Y38QftDbrLEzt0tAvFpIUEMvvJDd/BYJbQ='} 2023-03-20T09:52:25.181 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-20T09:52:25.187 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-03-20T09:52:27.680 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-03-20T09:52:27.680 INFO:teuthology.run_tasks:Running task console_log... 2023-03-20T09:52:27.767 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa3fda611f0>, signals=[15]) 2023-03-20T09:52:29.137 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-20T09:52:29.751 INFO:teuthology.task.internal:Opening connections... 2023-03-20T09:52:32.985 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T09:52:32.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:52:33.056 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T09:52:35.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:52:35.760 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-20T09:52:37.436 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2023-03-20T09:52:37.457 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2023-03-20T09:52:38.134 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2023-03-20T09:52:38.153 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2023-03-20T09:52:39.561 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8" 2023-03-20T09:52:39.561 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2023-03-20T09:52:39.561 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2023-03-20T09:52:39.561 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8" 2023-03-20T09:52:40.587 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2023-03-20T09:52:40.587 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-20T09:52:40.587 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2023-03-20T09:52:40.587 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-20T09:52:40.587 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2023-03-20T09:52:40.588 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-20T09:52:40.588 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-20T09:52:42.509 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-20T09:52:42.510 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2023-03-20T09:52:44.268 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2023-03-20T09:52:46.829 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2023-03-20T09:52:49.338 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2023-03-20T09:52:49.356 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2023-03-20T09:52:51.061 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8" 2023-03-20T09:52:51.061 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2023-03-20T09:52:51.061 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2023-03-20T09:52:51.061 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8" 2023-03-20T09:52:51.551 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8" 2023-03-20T09:52:51.551 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-20T09:52:52.359 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-20T09:52:52.359 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2023-03-20T09:52:52.382 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-20T09:52:53.509 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-20T09:52:55.656 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-20T09:52:56.507 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2023-03-20T09:52:57.436 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2023-03-20T09:52:57.453 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-20T09:52:58.425 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-20T09:53:00.278 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2023-03-20T09:53:00.280 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2023-03-20T09:53:01.832 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-20T09:53:03.411 INFO:teuthology.run_tasks:Running task kernel... 2023-03-20T09:53:04.977 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-03-20T09:53:06.976 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-03-20T09:53:06.977 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-20T09:53:06.978 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-20T09:53:06.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-20T09:53:09.566 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2023-03-20T09:53:09.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-20T09:53:09.567 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-03-20T09:53:09.624 INFO:teuthology.orchestra.run.smithi088.stdout:4.18.0-408.el8.x86_64 2023-03-20T09:53:11.113 INFO:teuthology.task.kernel:Running kernel on smithi088: 4.18.0-408.el8.x86_64 2023-03-20T09:53:11.113 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum install -y kernel 2023-03-20T09:53:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-408.el8.x86_64 2023-03-20T09:53:11.115 INFO:teuthology.task.kernel:Running kernel on smithi072: 4.18.0-408.el8.x86_64 2023-03-20T09:53:11.115 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2023-03-20T09:53:14.615 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - AppStream 38 MB/s | 29 MB 00:00 2023-03-20T09:53:16.730 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - AppStream 32 MB/s | 29 MB 00:00 2023-03-20T09:53:20.390 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - BaseOS 45 MB/s | 28 MB 00:00 2023-03-20T09:53:22.666 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - BaseOS 34 MB/s | 28 MB 00:00 2023-03-20T09:53:24.940 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - Extras 112 kB/s | 18 kB 00:00 2023-03-20T09:53:25.434 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - PowerTools 21 MB/s | 5.7 MB 00:00 2023-03-20T09:53:26.457 INFO:teuthology.orchestra.run.smithi088.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-03-20T09:53:27.900 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 14 MB/s | 14 MB 00:00 2023-03-20T09:53:27.900 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - Extras 143 kB/s | 18 kB 00:00 2023-03-20T09:53:27.900 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - PowerTools 18 MB/s | 5.7 MB 00:00 2023-03-20T09:53:28.761 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-03-20T09:53:30.563 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2023-03-20T09:53:30.882 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 465 kB/s | 24 kB 00:00 2023-03-20T09:53:33.355 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-03-20T09:53:34.163 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-20T09:53:35.041 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-20T09:53:35.043 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T09:53:35.043 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T09:53:36.201 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T09:53:36.201 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T09:53:36.201 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T09:53:36.692 INFO:teuthology.orchestra.run.smithi088.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-20T09:53:36.692 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-20T09:53:36.692 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-20T09:53:36.692 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:53:36.692 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T09:53:36.693 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T09:53:36.693 INFO:teuthology.orchestra.run.smithi088.stdout:Install 3 Packages 2023-03-20T09:53:37.992 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:53:37.992 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 85 M 2023-03-20T09:53:37.992 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 94 M 2023-03-20T09:53:37.992 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T09:53:37.992 INFO:teuthology.orchestra.run.smithi088.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 31 MB/s | 9.2 MB 00:00 2023-03-20T09:53:37.995 INFO:teuthology.orchestra.run.smithi088.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 34 MB/s | 41 MB 00:01 2023-03-20T09:53:37.995 INFO:teuthology.orchestra.run.smithi088.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-03-20T09:53:37.995 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T09:53:38.967 INFO:teuthology.orchestra.run.smithi088.stdout:Total 53 MB/s | 85 MB 00:01 2023-03-20T09:53:38.967 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T09:53:38.967 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T09:53:39.777 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T09:53:39.777 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-03-20T09:53:39.777 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-03-20T09:53:40.151 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T09:53:40.151 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:53:40.152 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T09:53:40.153 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T09:53:40.634 INFO:teuthology.orchestra.run.smithi072.stdout:Install 3 Packages 2023-03-20T09:53:41.605 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:53:41.605 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 85 M 2023-03-20T09:53:41.605 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 94 M 2023-03-20T09:53:41.605 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T09:53:41.606 INFO:teuthology.orchestra.run.smithi072.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 23 MB/s | 9.2 MB 00:00 2023-03-20T09:53:41.607 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T09:53:41.607 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T09:53:42.845 INFO:teuthology.orchestra.run.smithi072.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 30 MB/s | 41 MB 00:01 2023-03-20T09:53:42.845 INFO:teuthology.orchestra.run.smithi072.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 15 MB/s | 34 MB 00:02 2023-03-20T09:53:43.172 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T09:53:43.172 INFO:teuthology.orchestra.run.smithi072.stdout:Total 38 MB/s | 85 MB 00:02 2023-03-20T09:53:43.172 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T09:53:43.870 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T09:53:43.870 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T09:53:43.871 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T09:53:45.392 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T09:53:45.393 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T09:53:45.393 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T09:53:45.393 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:53:45.844 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:53:46.990 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:53:48.786 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:53:49.636 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:53:51.112 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:53:51.112 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:53:51.112 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:53:53.531 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:53:56.570 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:12.139 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:12.888 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:12.889 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:12.889 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:54:12.889 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:54:13.198 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:54:13.973 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T09:54:13.977 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel --last | head -n 1 2023-03-20T09:54:16.026 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-4.18.0-448.el8.x86_64 Mon 20 Mar 2023 09:53:50 AM UTC 2023-03-20T09:54:17.260 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:17.260 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:17.261 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-20T09:54:17.261 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-20T09:54:17.261 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-20T09:54:17.261 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-20T09:54:17.261 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-03-20T09:54:17.261 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=None, version=distro) 2023-03-20T09:54:17.261 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel --last | head -n 1 2023-03-20T09:54:17.448 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-4.18.0-448.el8.x86_64 Mon 20 Mar 2023 09:53:50 AM UTC 2023-03-20T09:54:19.916 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-20T09:54:19.916 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2023-03-20T09:54:19.916 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T09:54:19.917 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-03-20T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-03-20T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:54:21.551 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T09:54:21.551 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel --last | head -n 1 2023-03-20T09:54:22.448 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2023-03-20T09:54:24.624 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2023-03-20T09:54:24.624 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.02 2023-03-20T09:54:24.624 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 129.el8 2023-03-20T09:54:24.624 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2023-03-20T09:54:24.624 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Group : System Environment/Base 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 9265586 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Relocations : (not relocatable) 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : CentOS BuildSystem 2023-03-20T09:54:24.625 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2023-03-20T09:54:26.059 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T09:54:26.060 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2023-03-20T09:54:26.061 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-448.el8.x86_64 Mon 20 Mar 2023 09:53:53 AM UTC 2023-03-20T09:54:26.061 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-20T09:54:26.061 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 4.18.0-448.el8.x86_64 2023-03-20T09:54:26.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-20T09:54:26.062 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-20T09:54:26.062 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-03-20T09:54:26.062 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-03-20T09:54:26.062 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-03-20T09:54:26.062 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-03-20T09:54:26.062 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi072.front.sepia.ceph.com, path=None, version=distro) 2023-03-20T09:54:26.062 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel --last | head -n 1 2023-03-20T09:54:27.531 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-4.18.0-448.el8.x86_64 Mon 20 Mar 2023 09:53:53 AM UTC 2023-03-20T09:54:29.337 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2023-03-20T09:54:29.964 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-03-20T09:54:29.965 DEBUG:teuthology.orchestra.run.smithi072:> sudo rpm -qi grub2-tools 2023-03-20T09:54:30.004 INFO:teuthology.orchestra.run.smithi072.stdout:Name : grub2-tools 2023-03-20T09:54:31.154 INFO:teuthology.orchestra.run.smithi072.stdout:Epoch : 1 2023-03-20T09:54:31.154 INFO:teuthology.orchestra.run.smithi072.stdout:Version : 2.02 2023-03-20T09:54:31.154 INFO:teuthology.orchestra.run.smithi072.stdout:Release : 129.el8 2023-03-20T09:54:31.154 INFO:teuthology.orchestra.run.smithi072.stdout:Architecture: x86_64 2023-03-20T09:54:31.154 INFO:teuthology.orchestra.run.smithi072.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-20T09:54:31.965 INFO:teuthology.orchestra.run.smithi072.stdout:Group : System Environment/Base 2023-03-20T09:54:31.965 INFO:teuthology.orchestra.run.smithi072.stdout:Size : 9265586 2023-03-20T09:54:31.966 INFO:teuthology.orchestra.run.smithi072.stdout:License : GPLv3+ 2023-03-20T09:54:32.675 INFO:teuthology.orchestra.run.smithi072.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-20T09:54:32.675 INFO:teuthology.orchestra.run.smithi072.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-20T09:54:33.594 INFO:teuthology.orchestra.run.smithi072.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-20T09:54:33.595 INFO:teuthology.orchestra.run.smithi072.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-20T09:54:33.595 INFO:teuthology.orchestra.run.smithi072.stdout:Relocations : (not relocatable) 2023-03-20T09:54:33.595 INFO:teuthology.orchestra.run.smithi072.stdout:Packager : CentOS BuildSystem 2023-03-20T09:54:33.595 INFO:teuthology.orchestra.run.smithi072.stdout:Vendor : CentOS 2023-03-20T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-20T09:54:34.079 INFO:teuthology.orchestra.run.smithi072.stdout:Summary : Support tools for GRUB. 2023-03-20T09:54:34.484 INFO:teuthology.orchestra.run.smithi072.stdout:Description : 2023-03-20T09:54:35.660 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:54:35.660 INFO:teuthology.orchestra.run.smithi072.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-20T09:54:35.660 INFO:teuthology.orchestra.run.smithi072.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-20T09:54:35.660 INFO:teuthology.orchestra.run.smithi072.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-20T09:54:35.661 INFO:teuthology.orchestra.run.smithi072.stdout:hardware devices. 2023-03-20T09:54:35.661 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T09:54:35.661 INFO:teuthology.orchestra.run.smithi072.stdout:This subpackage provides tools for support of all platforms. 2023-03-20T09:54:35.662 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-20T09:54:37.173 INFO:teuthology.task.kernel:Updating grub on smithi072 to boot 4.18.0-448.el8.x86_64 2023-03-20T09:54:37.173 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-20T09:54:37.174 INFO:teuthology.orchestra.run.smithi088.stderr:done 2023-03-20T09:54:37.174 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2023-03-20T09:54:37.205 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.2P4SaCgm2M 2023-03-20T09:54:38.352 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2P4SaCgm2M 2023-03-20T09:54:38.353 INFO:teuthology.orchestra.run.smithi072.stderr:Generating grub configuration file ... 2023-03-20T09:54:38.408 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.2P4SaCgm2M 2023-03-20T09:54:39.842 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.2P4SaCgm2M is 6KB 2023-03-20T09:54:41.078 INFO:teuthology.orchestra.run.smithi072.stderr:done 2023-03-20T09:54:42.661 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2023-03-20T09:54:42.708 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.2P4SaCgm2M 2023-03-20T09:54:44.385 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.Rr1GkemYLu 2023-03-20T09:54:44.385 DEBUG:teuthology.orchestra.run.smithi072:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Rr1GkemYLu 2023-03-20T09:54:44.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2023-03-20T09:54:47.229 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.Rr1GkemYLu 2023-03-20T09:54:47.267 INFO:teuthology.orchestra.run.smithi088.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-20T09:54:49.819 INFO:teuthology.orchestra.run.smithi088.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-20T09:54:53.129 INFO:teuthology.orchestra.run.smithi088.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-20T09:54:53.130 INFO:teuthology.orchestra.run.smithi088.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-20T09:54:53.131 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-20T09:54:53.176 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.Rr1GkemYLu is 6KB 2023-03-20T09:54:55.333 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2023-03-20T09:54:55.379 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.Rr1GkemYLu 2023-03-20T09:54:56.879 DEBUG:teuthology.orchestra.run.smithi072:> sudo /bin/ls /boot/loader/entries || true 2023-03-20T09:54:59.270 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-20T09:55:00.736 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-20T09:55:00.736 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-20T09:55:00.736 INFO:teuthology.orchestra.run.smithi072.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-03-20T09:55:00.737 DEBUG:teuthology.orchestra.run.smithi072:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-03-20T09:55:00.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo shutdown -r now 2023-03-20T09:55:25.360 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-03-20T09:55:27.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-03-20T09:55:27.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:55:32.521 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-03-20T09:55:34.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-20T09:55:34.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:55:46.053 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-03-20T09:55:53.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.72 2023-03-20T09:56:18.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-03-20T09:56:19.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:56:24.418 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-20T09:56:25.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:56:38.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2023-03-20T09:56:44.953 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-03-20T09:56:46.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-20T09:56:49.836 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-03-20T09:56:49.837 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2023-03-20T09:56:49.855 INFO:teuthology.orchestra.run.smithi072.stdout:4.18.0-448.el8.x86_64 2023-03-20T09:56:52.807 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-20T09:56:53.435 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-20T09:56:53.435 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-20T09:56:54.436 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-03-20T09:56:54.867 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-20T09:56:55.213 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2023-03-20T09:56:55.738 DEBUG:teuthology.parallel:result is None 2023-03-20T09:57:10.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-03-20T09:57:14.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T09:57:14.951 DEBUG:teuthology.orchestra.run.smithi088:> true 2023-03-20T09:57:15.894 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-03-20T09:57:17.050 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-03-20T09:57:27.934 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2023-03-20T09:57:28.966 INFO:teuthology.orchestra.run.smithi088.stdout:4.18.0-448.el8.x86_64 2023-03-20T09:57:30.968 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-03-20T09:57:30.968 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-20T09:57:30.969 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-20T09:57:31.969 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-03-20T09:57:33.301 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-20T09:57:33.494 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2023-03-20T09:57:35.971 DEBUG:teuthology.parallel:result is None 2023-03-20T09:57:35.971 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-20T09:57:35.979 INFO:teuthology.task.internal:Creating test directory... 2023-03-20T09:57:36.985 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-20T09:57:37.478 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-20T09:57:38.880 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-20T09:57:40.779 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-20T09:57:42.515 INFO:teuthology.task.internal:Creating archive directory... 2023-03-20T09:57:51.932 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-20T09:57:51.936 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-20T09:57:53.021 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-20T09:57:53.889 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-20T09:57:59.856 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-20T09:58:01.048 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-20T09:58:02.536 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-20T09:58:02.537 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-20T09:58:02.615 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-20T09:58:04.357 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-20T09:58:16.052 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-20T09:58:16.060 INFO:teuthology.task.internal:Configuring sudo... 2023-03-20T09:58:16.668 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-20T09:58:16.670 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-20T09:58:17.928 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-20T09:58:19.214 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-20T09:58:21.491 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-20T09:58:21.495 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-20T09:58:23.721 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-20T09:58:25.533 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-20T09:58:26.747 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-20T09:58:29.171 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-20T09:58:31.830 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T09:58:33.025 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-20T09:58:33.055 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-20T09:58:34.729 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-20T09:58:35.373 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-20T09:58:38.934 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-20T09:58:39.690 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T09:58:40.805 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-20T09:58:40.836 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2023-03-20T09:58:42.380 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2023-03-20T09:58:42.441 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-20T09:58:43.690 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-20T09:58:43.690 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-20T09:58:43.699 INFO:teuthology.task.internal:Starting timer... 2023-03-20T09:58:53.299 INFO:teuthology.run_tasks:Running task pcp... 2023-03-20T09:58:53.315 INFO:teuthology.run_tasks:Running task selinux... 2023-03-20T09:58:55.356 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-20T09:58:57.105 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2023-03-20T09:58:57.171 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2023-03-20T09:58:57.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2023-03-20T09:58:58.437 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2023-03-20T09:58:59.670 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-20T09:58:59.670 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2023-03-20T09:59:00.714 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2023-03-20T09:59:12.990 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2023-03-20T09:59:14.922 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2023-03-20T09:59:16.541 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi088.front.sepia.ceph.com': 'permissive'} 2023-03-20T09:59:16.541 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-20T09:59:16.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-20T09:59:18.108 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-20T09:59:18.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-20T09:59:20.169 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-20T09:59:20.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2023-03-20T09:59:20.202 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2023-03-20T09:59:21.967 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-20T09:59:23.291 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-03-20T09:59:25.086 INFO:teuthology.repo_utils:Fetching main from origin 2023-03-20T09:59:27.049 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-20T09:59:28.580 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-20T09:59:29.765 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com,smithi088.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-20T10:06:35.028 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi088.front.sepia.ceph.com')] 2023-03-20T10:06:36.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2023-03-20T10:06:38.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T10:06:38.536 DEBUG:teuthology.orchestra.run.smithi072:> true 2023-03-20T10:06:40.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2023-03-20T10:06:43.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-20T10:06:43.113 DEBUG:teuthology.orchestra.run.smithi088:> true 2023-03-20T10:06:55.976 INFO:teuthology.run_tasks:Running task clock... 2023-03-20T10:06:57.403 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-20T10:06:58.739 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-20T10:06:58.740 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-20T10:06:58.744 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-20T10:07:02.466 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-20T10:07:02.467 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-20T10:07:02.467 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-20T10:07:02.467 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2023-03-20T10:07:02.467 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2023-03-20T10:07:02.468 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-20T10:07:02.468 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-20T10:07:02.468 INFO:teuthology.orchestra.run.smithi072.stderr:bash: ntpq: command not found 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:02.469 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:02.498 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-20T10:07:03.732 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-20T10:07:03.733 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2023-03-20T10:07:03.733 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2023-03-20T10:07:03.734 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-20T10:07:03.734 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-20T10:07:04.230 INFO:teuthology.orchestra.run.smithi088.stderr:bash: ntpq: command not found 2023-03-20T10:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-20T10:07:04.231 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2023-03-20T10:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:04.232 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-20T10:07:04.232 INFO:teuthology.run_tasks:Running task install... 2023-03-20T10:07:04.241 DEBUG:teuthology.task.install:project ceph 2023-03-20T10:07:06.203 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c'}} 2023-03-20T10:07:07.981 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c'} 2023-03-20T10:07:07.981 INFO:teuthology.task.install:Using flavor: default 2023-03-20T10:07:07.988 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-20T10:07:09.316 INFO:teuthology.task.install:extra packages: [] 2023-03-20T10:07:09.316 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c', 'tag': None, 'wait_for_package': False} 2023-03-20T10:07:10.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6dd66830c12498d02f3894e352dd8b49a7bab4c 2023-03-20T10:07:10.085 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c', 'tag': None, 'wait_for_package': False} 2023-03-20T10:07:11.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6dd66830c12498d02f3894e352dd8b49a7bab4c 2023-03-20T10:07:11.855 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/ 2023-03-20T10:07:13.725 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2743.ga6dd6683 2023-03-20T10:07:13.731 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/ 2023-03-20T10:07:15.117 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2743.ga6dd6683 2023-03-20T10:07:15.203 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-20T10:07:16.511 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:07:16.511 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-20T10:07:16.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/a6dd66830c12498d02f3894e352dd8b49a7bab4c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-20T10:07:17.134 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:07:17.134 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-20T10:07:17.165 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-03-20T10:07:18.592 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6dd66830c12498d02f3894e352dd8b49a7bab4c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-20T10:07:18.594 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-03-20T10:07:18.594 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6dd66830c12498d02f3894e352dd8b49a7bab4c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-20T10:07:18.625 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-20T10:07:20.260 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-20T10:07:21.174 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-20T10:07:31.021 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2023-03-20T10:07:32.722 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-20T10:07:32.723 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2023-03-20T10:07:32.795 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2023-03-20T10:07:34.143 INFO:teuthology.orchestra.run.smithi088.stdout:71 files removed 2023-03-20T10:07:34.143 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2023-03-20T10:07:34.143 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw 2023-03-20T10:07:35.252 INFO:teuthology.orchestra.run.smithi072.stdout:71 files removed 2023-03-20T10:07:36.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 307 kB/s | 80 kB 00:00 2023-03-20T10:07:36.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 37 kB/s | 9.3 kB 00:00 2023-03-20T10:07:36.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-03-20T10:07:37.904 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw 2023-03-20T10:07:37.995 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - AppStream 17 MB/s | 29 MB 00:01 2023-03-20T10:07:43.470 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 310 kB/s | 80 kB 00:00 2023-03-20T10:07:44.646 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 41 kB/s | 9.3 kB 00:00 2023-03-20T10:07:44.647 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-03-20T10:07:44.647 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - AppStream 33 MB/s | 29 MB 00:00 2023-03-20T10:07:44.647 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - BaseOS 45 MB/s | 28 MB 00:00 2023-03-20T10:07:45.948 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - BaseOS 35 MB/s | 28 MB 00:00 2023-03-20T10:07:48.077 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - Extras 142 kB/s | 18 kB 00:00 2023-03-20T10:07:48.821 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-8 - PowerTools 23 MB/s | 5.7 MB 00:00 2023-03-20T10:07:49.878 INFO:teuthology.orchestra.run.smithi088.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2023-03-20T10:07:50.405 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - Extras 155 kB/s | 18 kB 00:00 2023-03-20T10:07:50.759 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS-8 - PowerTools 17 MB/s | 5.7 MB 00:00 2023-03-20T10:07:52.303 INFO:teuthology.orchestra.run.smithi072.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-03-20T10:07:53.637 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 14 MB 00:03 2023-03-20T10:07:57.164 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 14 MB 00:02 2023-03-20T10:07:57.164 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-03-20T10:07:58.304 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 381 kB/s | 24 kB 00:00 2023-03-20T10:08:00.573 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:08:02.010 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:08:02.010 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 13 M 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-03-20T10:08:02.011 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.7 M 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.6 M 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 5.4 M 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 22 M 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 24 k 2023-03-20T10:08:02.686 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 768 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 474 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 6.5 M 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 45 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 117 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 190 k 2023-03-20T10:08:03.939 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-20T10:08:05.845 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 371 k 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 365 k 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 106 k 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:Install 21 Packages 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 2 Packages 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 59 M 2023-03-20T10:08:06.496 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:08:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 13 M 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.7 M 2023-03-20T10:08:08.242 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.6 M 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 5.4 M 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 22 M 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 24 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 768 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 474 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 6.5 M 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse 2023-03-20T10:08:08.243 INFO:teuthology.orchestra.run.smithi088.stdout: x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 45 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 117 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 190 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 371 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 365 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 106 k 2023-03-20T10:08:09.996 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:Install 21 Packages 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 2 Packages 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 59 M 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(1/23): ceph-base-18.0.0-2743.ga6dd6683.el8.x86 6.4 MB/s | 5.4 MB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(2/23): ceph-selinux-18.0.0-2743.ga6dd6683.el8. 419 kB/s | 24 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(3/23): libcephfs2-18.0.0-2743.ga6dd6683.el8.x8 5.0 MB/s | 768 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(4/23): libradosstriper1-18.0.0-2743.ga6dd6683. 5.0 MB/s | 474 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(5/23): ceph-radosgw-18.0.0-2743.ga6dd6683.el8. 9.7 MB/s | 13 MB 00:01 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(6/23): python3-ceph-argparse-18.0.0-2743.ga6dd 671 kB/s | 45 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(7/23): python3-ceph-common-18.0.0-2743.ga6dd66 1.5 MB/s | 117 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(8/23): python3-cephfs-18.0.0-2743.ga6dd6683.el 2.5 MB/s | 190 kB 00:00 2023-03-20T10:08:09.997 INFO:teuthology.orchestra.run.smithi088.stdout:(9/23): python3-rados-18.0.0-2743.ga6dd6683.el8 4.3 MB/s | 371 kB 00:00 2023-03-20T10:08:09.998 INFO:teuthology.orchestra.run.smithi088.stdout:(10/23): python3-rbd-18.0.0-2743.ga6dd6683.el8. 4.3 MB/s | 365 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(11/23): librgw2-18.0.0-2743.ga6dd6683.el8.x86_ 9.6 MB/s | 6.5 MB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(12/23): python3-rgw-18.0.0-2743.ga6dd6683.el8. 983 kB/s | 106 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(15/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 931 kB/s | 47 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(17/23): ceph-common-18.0.0-2743.ga6dd6683.el8. 9.6 MB/s | 22 MB 00:02 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 207 kB/s | 59 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 748 kB/s | 306 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 300 kB/s | 75 kB 00:00 2023-03-20T10:08:11.716 INFO:teuthology.orchestra.run.smithi088.stdout:(21/23): librados2-18.0.0-2743.ga6dd6683.el8.x8 5.8 MB/s | 3.7 MB 00:00 2023-03-20T10:08:13.750 INFO:teuthology.orchestra.run.smithi088.stdout:(22/23): thrift-0.13.0-2.el8.x86_64.rpm 2.0 MB/s | 1.7 MB 00:00 2023-03-20T10:08:13.750 INFO:teuthology.orchestra.run.smithi088.stdout:(23/23): librbd1-18.0.0-2743.ga6dd6683.el8.x86_ 4.4 MB/s | 3.6 MB 00:00 2023-03-20T10:08:13.750 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:08:13.750 INFO:teuthology.orchestra.run.smithi088.stdout:Total 18 MB/s | 59 MB 00:03 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-03-20T10:08:13.751 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-03-20T10:08:13.755 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/25 2023-03-20T10:08:14.552 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:15.940 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:15.940 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-03-20T10:08:15.941 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-20T10:08:18.056 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-20T10:08:18.056 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8. 10/25 2023-03-20T10:08:18.056 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 11/25 2023-03-20T10:08:18.056 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 12/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 13/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 13/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 16/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 16/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 17/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x8 19/25 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi072.stdout:(1/23): ceph-common-18.0.0-2743.ga6dd6683.el8.x 12 MB/s | 22 MB 00:01 2023-03-20T10:08:18.057 INFO:teuthology.orchestra.run.smithi072.stdout:(2/23): ceph-selinux-18.0.0-2743.ga6dd6683.el8. 584 kB/s | 24 kB 00:00 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(3/23): libcephfs2-18.0.0-2743.ga6dd6683.el8.x8 8.0 MB/s | 768 kB 00:00 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(4/23): ceph-base-18.0.0-2743.ga6dd6683.el8.x86 2.6 MB/s | 5.4 MB 00:02 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(5/23): libradosstriper1-18.0.0-2743.ga6dd6683. 2.8 MB/s | 474 kB 00:00 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(6/23): python3-ceph-argparse-18.0.0-2743.ga6dd 886 kB/s | 45 kB 00:00 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(7/23): python3-ceph-common-18.0.0-2743.ga6dd66 2.0 MB/s | 117 kB 00:00 2023-03-20T10:08:19.092 INFO:teuthology.orchestra.run.smithi072.stdout:(8/23): python3-cephfs-18.0.0-2743.ga6dd6683.el 3.2 MB/s | 190 kB 00:00 2023-03-20T10:08:19.093 INFO:teuthology.orchestra.run.smithi072.stdout:(9/23): python3-rados-18.0.0-2743.ga6dd6683.el8 5.4 MB/s | 371 kB 00:00 2023-03-20T10:08:19.093 INFO:teuthology.orchestra.run.smithi072.stdout:(10/23): python3-rbd-18.0.0-2743.ga6dd6683.el8. 4.7 MB/s | 365 kB 00:00 2023-03-20T10:08:19.093 INFO:teuthology.orchestra.run.smithi072.stdout:(11/23): python3-rgw-18.0.0-2743.ga6dd6683.el8. 1.8 MB/s | 106 kB 00:00 2023-03-20T10:08:19.093 INFO:teuthology.orchestra.run.smithi072.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(13/23): librgw2-18.0.0-2743.ga6dd6683.el8.x86_ 11 MB/s | 6.5 MB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.2 MB/s | 47 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 518 kB/s | 44 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 219 kB/s | 59 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 499 kB/s | 75 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(19/23): gperftools-libs-2.7-9.el8.x86_64.rpm 654 kB/s | 306 kB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(20/23): librados2-18.0.0-2743.ga6dd6683.el8.x8 13 MB/s | 3.7 MB 00:00 2023-03-20T10:08:27.101 INFO:teuthology.orchestra.run.smithi072.stdout:(21/23): ceph-radosgw-18.0.0-2743.ga6dd6683.el8 3.4 MB/s | 13 MB 00:03 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:(22/23): librbd1-18.0.0-2743.ga6dd6683.el8.x86_ 8.3 MB/s | 3.6 MB 00:00 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.7 MB/s | 1.7 MB 00:01 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:Total 14 MB/s | 59 MB 00:04 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:08:27.102 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-03-20T10:08:27.862 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:27.863 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:29.468 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/25 2023-03-20T10:08:29.468 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/25 2023-03-20T10:08:29.468 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8. 10/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 11/25 2023-03-20T10:08:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 12/25 2023-03-20T10:08:29.470 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 13/25 2023-03-20T10:08:29.470 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 13/25 2023-03-20T10:08:29.470 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-03-20T10:08:30.043 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-03-20T10:08:30.043 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 16/25 2023-03-20T10:08:30.043 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 16/25 2023-03-20T10:08:30.043 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 17/25 2023-03-20T10:08:30.045 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 21/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 21/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x8 19/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 20/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 21/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 21/25 2023-03-20T10:08:37.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:37.508 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:39.703 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2023-03-20T10:08:39.704 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2023-03-20T10:08:39.704 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /dev 2023-03-20T10:08:39.704 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /run 2023-03-20T10:08:39.704 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /dev 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:08:41.336 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 23/25 2023-03-20T10:08:42.808 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 23/25 2023-03-20T10:08:42.808 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-20T10:08:42.808 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-20T10:08:42.808 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:42.813 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:44.084 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/25 2023-03-20T10:08:44.853 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/25 2023-03-20T10:08:44.853 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 3/25 2023-03-20T10:08:44.853 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:44.853 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 6/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8. 8/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x8 9/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 10/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 11/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 12/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 13/25 2023-03-20T10:08:44.854 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-03-20T10:08:44.855 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 24/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.624 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.625 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2023-03-20T10:08:46.626 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2023-03-20T10:08:48.042 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-03-20T10:08:48.042 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /run 2023-03-20T10:08:48.042 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2023-03-20T10:08:48.042 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2023-03-20T10:08:48.042 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /dev 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 23/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 23/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:48.043 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/25 2023-03-20T10:08:48.044 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/25 2023-03-20T10:08:48.044 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 3/25 2023-03-20T10:08:48.044 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/25 2023-03-20T10:08:48.044 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_6 6/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 7/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8. 8/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x8 9/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 10/25 2023-03-20T10:08:48.618 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 11/25 2023-03-20T10:08:51.124 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 12/25 2023-03-20T10:08:51.124 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 13/25 2023-03-20T10:08:51.124 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-03-20T10:08:51.124 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-03-20T10:08:51.125 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-03-20T10:08:51.125 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-03-20T10:08:51.125 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-03-20T10:08:51.125 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-03-20T10:08:51.125 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-03-20T10:08:52.178 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-03-20T10:08:52.178 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 22/25 2023-03-20T10:08:52.178 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-03-20T10:08:52.179 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 24/25 2023-03-20T10:08:52.179 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-test 2023-03-20T10:08:52.180 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-03-20T10:08:52.181 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:08:52.181 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-03-20T10:08:52.181 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:52.997 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:52.997 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:08:52.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:52.997 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.119 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-20T10:08:57.135 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.136 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-20T10:08:57.707 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-20T10:08:57.707 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.707 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:57.707 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:58.143 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:08:58.144 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:08:58.145 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-test 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:56 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 47 M 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-20T10:08:59.185 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 137 k 2023-03-20T10:08:59.186 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-20T10:08:59.186 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-20T10:08:59.186 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-20T10:08:59.186 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 48 M 2023-03-20T10:09:00.564 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 211 M 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.6 MB/s | 187 kB 00:00 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.5 MB/s | 323 kB 00:00 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): libcephsqlite-18.0.0-2743.ga6dd6683.el8. 540 kB/s | 137 kB 00:00 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): ceph-test-18.0.0-2743.ga6dd6683.el8.x86_ 19 MB/s | 47 MB 00:02 2023-03-20T10:09:00.565 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Total 19 MB/s | 48 MB 00:02 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-03-20T10:09:00.566 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-03-20T10:09:03.423 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-03-20T10:09:03.423 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-03-20T10:09:03.424 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-03-20T10:09:03.424 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/6 2023-03-20T10:09:03.425 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/6 2023-03-20T10:09:03.425 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/6 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:01 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:09:03.426 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 47 M 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 137 k 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 48 M 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 211 M 2023-03-20T10:09:03.427 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:09:03.428 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.9 MB/s | 202 kB 00:00 2023-03-20T10:09:04.519 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.5 MB/s | 187 kB 00:00 2023-03-20T10:09:04.520 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.4 MB/s | 323 kB 00:00 2023-03-20T10:09:04.520 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-03-20T10:09:04.520 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): libcephsqlite-18.0.0-2743.ga6dd6683.el8. 675 kB/s | 137 kB 00:00 2023-03-20T10:09:04.522 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): ceph-test-18.0.0-2743.ga6dd6683.el8.x86_ 18 MB/s | 47 MB 00:02 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Total 19 MB/s | 48 MB 00:02 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:09:05.701 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:09:05.702 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/6 2023-03-20T10:09:05.702 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/6 2023-03-20T10:09:05.702 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/6 2023-03-20T10:09:05.702 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-03-20T10:09:05.702 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-03-20T10:09:06.390 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-03-20T10:09:07.010 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: jq-1.6-6.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:07.011 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:09:07.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph 2023-03-20T10:09:07.013 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-03-20T10:09:07.013 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-03-20T10:09:07.013 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-03-20T10:09:07.014 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-03-20T10:09:07.014 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-03-20T10:09:08.009 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/6 2023-03-20T10:09:08.010 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:11 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:09:08.968 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:09:09.996 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2023-03-20T10:09:11.018 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:09:11.018 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2023-03-20T10:09:11.018 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:09:11.018 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 6.4 k 2023-03-20T10:09:11.018 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 2.3 M 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 1.5 M 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 236 k 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 4.6 M 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 18 M 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-20T10:09:11.019 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-20T10:09:11.613 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-20T10:09:11.614 INFO:teuthology.orchestra.run.smithi088.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-20T10:09:12.032 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-20T10:09:12.859 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-20T10:09:12.859 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-20T10:09:12.859 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-20T10:09:12.859 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:12.859 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:09:14.490 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2023-03-20T10:09:14.490 INFO:teuthology.orchestra.run.smithi088.stdout:Install 32 Packages 2023-03-20T10:09:14.490 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:15.036 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 30 M 2023-03-20T10:09:15.036 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 119 M 2023-03-20T10:09:15.036 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:09:15.036 INFO:teuthology.orchestra.run.smithi088.stdout:(1/32): ceph-18.0.0-2743.ga6dd6683.el8.x86_64.r 47 kB/s | 6.4 kB 00:00 2023-03-20T10:09:15.037 INFO:teuthology.orchestra.run.smithi088.stdout:(2/32): ceph-mds-18.0.0-2743.ga6dd6683.el8.x86_ 6.7 MB/s | 2.3 MB 00:00 2023-03-20T10:09:15.445 INFO:teuthology.orchestra.run.smithi088.stdout:(3/32): ceph-mgr-18.0.0-2743.ga6dd6683.el8.x86_ 4.2 MB/s | 1.5 MB 00:00 2023-03-20T10:09:15.445 INFO:teuthology.orchestra.run.smithi088.stdout:(4/32): ceph-mgr-modules-core-18.0.0-2743.ga6dd 2.8 MB/s | 236 kB 00:00 2023-03-20T10:09:15.445 INFO:teuthology.orchestra.run.smithi088.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 2.3 MB/s | 39 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 3.0 MB/s | 103 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 3.1 MB/s | 54 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-03-20T10:09:16.000 INFO:teuthology.orchestra.run.smithi088.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-03-20T10:09:16.418 INFO:teuthology.orchestra.run.smithi088.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 249 kB/s | 44 kB 00:00 2023-03-20T10:09:16.419 INFO:teuthology.orchestra.run.smithi088.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.3 MB/s | 185 kB 00:00 2023-03-20T10:09:16.421 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/6 2023-03-20T10:09:17.324 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 6/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: jq-1.6-6.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:17.325 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:09:17.326 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph 2023-03-20T10:09:17.328 INFO:teuthology.orchestra.run.smithi088.stdout:(15/32): ceph-osd-18.0.0-2743.ga6dd6683.el8.x86 20 MB/s | 18 MB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 738 kB/s | 173 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(18/32): ceph-mon-18.0.0-2743.ga6dd6683.el8.x86 3.6 MB/s | 4.6 MB 00:01 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 140 kB/s | 18 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 704 kB/s | 59 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.1 MB/s | 384 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 210 kB/s | 49 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 62 kB/s | 19 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 53 kB/s | 24 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(26/32): python3-pecan-1.3.2-9.el8.noarch.rpm 392 kB/s | 283 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 57 kB/s | 29 kB 00:00 2023-03-20T10:09:17.329 INFO:teuthology.orchestra.run.smithi088.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 64 kB/s | 27 kB 00:00 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 725 kB/s | 241 kB 00:00 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 537 kB/s | 85 kB 00:00 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:(32/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 910 kB/s | 251 kB 00:00 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:Total 11 MB/s | 30 MB 00:02 2023-03-20T10:09:17.330 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:09:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-03-20T10:09:19.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-03-20T10:09:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-03-20T10:09:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-03-20T10:09:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-03-20T10:09:20.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-20T10:09:20.032 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-03-20T10:09:20.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-03-20T10:09:20.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-03-20T10:09:20.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-03-20T10:09:20.569 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-03-20T10:09:21.440 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-03-20T10:09:21.440 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-03-20T10:09:21.440 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-03-20T10:09:21.440 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-03-20T10:09:21.440 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8. 27/32 2023-03-20T10:09:21.442 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 28/32 2023-03-20T10:09:21.444 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:20 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:09:23.988 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 6.4 k 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 2.3 M 2023-03-20T10:09:24.933 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 1.5 M 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 236 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 4.6 M 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 18 M 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-20T10:09:24.934 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-20T10:09:24.935 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-20T10:09:25.754 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-20T10:09:25.754 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-20T10:09:25.754 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-20T10:09:25.755 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout:Install 32 Packages 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 30 M 2023-03-20T10:09:27.144 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 119 M 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(1/32): ceph-18.0.0-2743.ga6dd6683.el8.x86_64.r 53 kB/s | 6.4 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(2/32): ceph-mds-18.0.0-2743.ga6dd6683.el8.x86_ 6.5 MB/s | 2.3 MB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(3/32): ceph-mgr-18.0.0-2743.ga6dd6683.el8.x86_ 3.9 MB/s | 1.5 MB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(4/32): ceph-mgr-modules-core-18.0.0-2743.ga6dd 2.5 MB/s | 236 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(5/32): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-03-20T10:09:28.134 INFO:teuthology.orchestra.run.smithi072.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 6.7 MB/s | 457 kB 00:00 2023-03-20T10:09:33.332 INFO:teuthology.orchestra.run.smithi072.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.2 MB/s | 237 kB 00:00 2023-03-20T10:09:33.332 INFO:teuthology.orchestra.run.smithi072.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 13 MB/s | 559 kB 00:00 2023-03-20T10:09:34.336 INFO:teuthology.orchestra.run.smithi072.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.2 MB/s | 109 kB 00:00 2023-03-20T10:09:34.336 INFO:teuthology.orchestra.run.smithi072.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 261 kB/s | 44 kB 00:00 2023-03-20T10:09:34.336 INFO:teuthology.orchestra.run.smithi072.stdout:(14/32): ceph-osd-18.0.0-2743.ga6dd6683.el8.x86 17 MB/s | 18 MB 00:01 2023-03-20T10:09:34.336 INFO:teuthology.orchestra.run.smithi072.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 435 kB/s | 185 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(16/32): ceph-mon-18.0.0-2743.ga6dd6683.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 929 kB/s | 173 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 209 kB/s | 18 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.3 MB/s | 384 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 643 kB/s | 49 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 580 kB/s | 59 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 308 kB/s | 16 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 317 kB/s | 19 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 405 kB/s | 24 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 494 kB/s | 29 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 465 kB/s | 27 kB 00:00 2023-03-20T10:09:35.788 INFO:teuthology.orchestra.run.smithi072.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-03-20T10:09:36.405 INFO:teuthology.orchestra.run.smithi072.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2023-03-20T10:09:36.406 INFO:teuthology.orchestra.run.smithi072.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.5 MB/s | 241 kB 00:00 2023-03-20T10:09:36.406 INFO:teuthology.orchestra.run.smithi072.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.6 MB/s | 251 kB 00:00 2023-03-20T10:09:36.406 INFO:teuthology.orchestra.run.smithi072.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 296 kB/s | 23 kB 00:00 2023-03-20T10:09:36.406 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:37.842 INFO:teuthology.orchestra.run.smithi072.stdout:Total 14 MB/s | 30 MB 00:02 2023-03-20T10:09:37.842 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:09:37.849 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 28/32 2023-03-20T10:09:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 29/32 2023-03-20T10:09:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 29/32 2023-03-20T10:09:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 30/32 2023-03-20T10:09:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 30/32 2023-03-20T10:09:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 31/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 31/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 32/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 32/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 3/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8. 6/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-03-20T10:09:39.459 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-03-20T10:09:39.460 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-20T10:09:39.461 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-2.6-1.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-20T10:09:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:42.628 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:09:42.629 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-base 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-03-20T10:09:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-03-20T10:09:43.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-03-20T10:09:43.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2023-03-20T10:09:44.426 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8. 27/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 28/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 28/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 29/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 29/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 30/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 30/32 2023-03-20T10:09:44.427 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 31/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 31/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 32/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 32/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/32 2023-03-20T10:09:44.757 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 3/32 2023-03-20T10:09:44.758 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 4/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 5/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8. 6/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-03-20T10:09:45.595 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-03-20T10:09:45.596 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:09:45.597 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:46.113 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-20T10:09:46.114 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-2.6-1.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-20T10:09:46.115 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-20T10:09:46.986 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-20T10:09:46.986 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:09:46.986 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:09:46.986 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-base 2023-03-20T10:09:46.988 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:47 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:09:46.988 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:09:46.988 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:09:46.988 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:09:46.988 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:09:46.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install cephadm 2023-03-20T10:09:47.714 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:49 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:09:51.946 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-base-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:09:53.744 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:09:53.744 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:09:53.744 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:09:53.745 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install cephadm 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:51 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 83 k 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:54.866 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 83 k 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 377 k 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:cephadm-18.0.0-2743.ga6dd6683.el8.noarch.rpm 285 kB/s | 83 kB 00:00 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Total 284 kB/s | 83 kB 00:00 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:09:54.867 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:09:55.451 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:09:55.451 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:09:55.451 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:09:55.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:56.101 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:09:56.101 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:09:56.101 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:56.101 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:09:56.102 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-immutable-object-cache 2023-03-20T10:09:56.103 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:57 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:09:56.773 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:09:58.306 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:00 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 165 k 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:09:59.811 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 165 k 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 503 k 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-immutable-object-cache-18.0.0-2743.ga6dd66 858 kB/s | 165 kB 00:00 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Total 853 kB/s | 165 kB 00:00 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:09:59.812 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:09:59.813 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:10:02.259 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 83 k 2023-03-20T10:10:02.259 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:02.259 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 83 k 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 377 k 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm-18.0.0-2743.ga6dd6683.el8.noarch.rpm 510 kB/s | 83 kB 00:00 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Total 506 kB/s | 83 kB 00:00 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:10:02.260 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:10:02.262 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:10:03.771 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:03.772 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:10:03.773 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:10:03.773 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:03.773 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:10:03.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-immutable-object-cache 2023-03-20T10:10:03.773 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr 2023-03-20T10:10:05.902 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:07 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:10:07.328 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:09 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:10:07.329 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:10:08.554 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:10:08.554 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:10:08.554 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:10:08.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-dashboard 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 165 k 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:08.556 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 165 k 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 503 k 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:10:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-immutable-object-cache-18.0.0-2743.ga6dd66 888 kB/s | 165 kB 00:00 2023-03-20T10:10:09.987 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:09.987 INFO:teuthology.orchestra.run.smithi072.stdout:Total 882 kB/s | 165 kB 00:00 2023-03-20T10:10:09.987 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:10:09.987 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:09.988 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:09.990 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:13 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:10:09.990 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683. 1/1 2023-03-20T10:10:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:09.991 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:10:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:10:09.991 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:09.991 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:09.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr 2023-03-20T10:10:10.493 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:10:11.024 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:12 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:10:11.026 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:10:11.026 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:10:11.026 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:10:11.027 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:10:11.640 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 1.7 M 2023-03-20T10:10:11.640 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:10:11.641 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 23 k 2023-03-20T10:10:11.641 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 15 k 2023-03-20T10:10:11.641 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-20T10:10:11.641 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-20T10:10:11.641 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-20T10:10:14.292 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:14.905 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:10:14.905 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 2.0 M 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 8.2 M 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): ceph-prometheus-alerts-18.0.0-2743.ga6dd 102 kB/s | 15 kB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): ceph-grafana-dashboards-18.0.0-2743.ga6d 144 kB/s | 23 kB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): ceph-mgr-dashboard-18.0.0-2743.ga6dd6683 6.2 MB/s | 1.7 MB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 251 kB/s | 34 kB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2023-03-20T10:10:14.906 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:10:15.876 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:10:15.876 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:10:15.876 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:10:15.876 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:10:15.878 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-mgr-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:10:15.878 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:10:15.878 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:10:15.879 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:15.879 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-dashboard 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.n 4/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8. 5/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 6/6 2023-03-20T10:10:15.880 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 6/6 2023-03-20T10:10:16.367 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8. 1/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 2/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.n 3/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:10:16.368 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:17.569 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:19.491 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:23.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-20T10:10:23.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-20T10:10:23.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-20T10:10:23.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:23.177 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:10:23.177 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-diskprediction-local 2023-03-20T10:10:23.899 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:25 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:10:24.186 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:27 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:10:25.110 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:10:25.828 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout:===================================================================================================== 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout:===================================================================================================== 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 7.4 M 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-20T10:10:26.889 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:===================================================================================================== 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:Install 8 Packages 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 35 M 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 210 M 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 18 MB/s | 4.9 MB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.7 MB/s | 225 kB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 18 MB/s | 3.7 MB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 6.7 MB/s | 172 kB 00:00 2023-03-20T10:10:26.890 INFO:teuthology.orchestra.run.smithi088.stdout:(7/8): ceph-mgr-diskprediction-local-18.0.0-274 13 MB/s | 7.4 MB 00:00 2023-03-20T10:10:26.892 INFO:teuthology.orchestra.run.smithi088.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 26 MB/s | 14 MB 00:00 2023-03-20T10:10:29.222 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Total 42 MB/s | 35 MB 00:00 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:10:29.223 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 1.7 M 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 23 k 2023-03-20T10:10:29.225 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 15 k 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 2.0 M 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 8.2 M 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): ceph-grafana-dashboards-18.0.0-2743.ga6d 153 kB/s | 23 kB 00:00 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): ceph-prometheus-alerts-18.0.0-2743.ga6dd 83 kB/s | 15 kB 00:00 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 453 kB/s | 43 kB 00:00 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 220 kB/s | 34 kB 00:00 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): ceph-mgr-dashboard-18.0.0-2743.ga6dd6683 4.5 MB/s | 1.7 MB 00:00 2023-03-20T10:10:29.226 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Total 4.6 MB/s | 2.0 MB 00:00 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.n 4/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8. 5/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 6/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 6/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8. 1/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarc 2/6 2023-03-20T10:10:29.227 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.n 3/6 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:29.228 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:29.228 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-diskprediction-local 2023-03-20T10:10:29.873 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:10:29.873 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-20T10:10:29.873 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-20T10:10:29.873 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-20T10:10:31.495 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-20T10:10:31.496 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:31.496 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:31.496 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-20T10:10:32.316 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-20T10:10:32.316 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-03-20T10:10:32.316 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-20T10:10:32.316 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-20T10:10:33.060 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:35 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:10:34.036 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 8/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 8/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 1/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-03-20T10:10:34.037 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-03-20T10:10:34.258 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:10:35.670 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-03-20T10:10:36.285 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:36.285 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:10:36.286 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:36.286 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-20T10:10:36.844 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:38.304 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:10:38.305 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-rook 2023-03-20T10:10:38.307 INFO:teuthology.orchestra.run.smithi072.stdout:===================================================================================================== 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout:===================================================================================================== 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 7.4 M 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-20T10:10:39.780 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:===================================================================================================== 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:Install 8 Packages 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 35 M 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 210 M 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 22 MB/s | 4.9 MB 00:00 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 14 MB/s | 3.7 MB 00:00 2023-03-20T10:10:39.781 INFO:teuthology.orchestra.run.smithi072.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 15 MB/s | 645 kB 00:00 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:(7/8): ceph-mgr-diskprediction-local-18.0.0-274 10 MB/s | 7.4 MB 00:00 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 20 MB/s | 14 MB 00:00 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Total 33 MB/s | 35 MB 00:01 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:10:40.513 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-03-20T10:10:40.514 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:40.516 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:42.717 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-20T10:10:42.717 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-03-20T10:10:42.717 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-03-20T10:10:42.717 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-20T10:10:42.718 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:02:43 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 47 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-20T10:10:42.719 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================== 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:Install 13 Packages 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 1.9 M 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 25 M 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 494 kB/s | 27 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(2/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.6 MB/s | 126 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 154 kB/s | 20 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(6/13): ceph-mgr-rook-18.0.0-2743.ga6dd6683.el8 230 kB/s | 47 kB 00:00 2023-03-20T10:10:42.720 INFO:teuthology.orchestra.run.smithi088.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 564 kB/s | 43 kB 00:00 2023-03-20T10:10:42.721 INFO:teuthology.orchestra.run.smithi088.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 7.3 kB/s | 16 kB 00:02 2023-03-20T10:10:42.721 INFO:teuthology.orchestra.run.smithi088.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 14 kB/s | 33 kB 00:02 2023-03-20T10:10:42.721 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-03-20T10:10:42.731 INFO:teuthology.orchestra.run.smithi088.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 36 kB/s | 81 kB 00:02 2023-03-20T10:10:43.335 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-03-20T10:10:44.491 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 8/8 2023-03-20T10:10:44.492 INFO:teuthology.orchestra.run.smithi088.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 613 kB/s | 61 kB 00:00 2023-03-20T10:10:44.492 INFO:teuthology.orchestra.run.smithi088.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 610 kB/s | 61 kB 00:00 2023-03-20T10:10:44.493 INFO:teuthology.orchestra.run.smithi088.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.7 MB/s | 1.2 MB 00:00 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Total 686 kB/s | 1.9 MB 00:02 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-03-20T10:10:44.497 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-03-20T10:10:44.499 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 8/8 2023-03-20T10:10:44.499 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd668 1/8 2023-03-20T10:10:44.499 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-03-20T10:10:45.368 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-20T10:10:45.369 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-20T10:10:45.370 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-20T10:10:45.370 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-20T10:10:45.370 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-20T10:10:45.370 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:45.370 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:45.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-rook 2023-03-20T10:10:45.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-03-20T10:10:45.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-03-20T10:10:45.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-03-20T10:10:45.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-03-20T10:10:45.503 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-03-20T10:10:46.164 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:48 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:10:46.517 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-03-20T10:10:47.433 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-03-20T10:10:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-03-20T10:10:49.721 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-03-20T10:10:49.724 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:10:49.724 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================== 2023-03-20T10:10:49.724 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:10:49.724 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================== 2023-03-20T10:10:49.725 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:10:49.725 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 47 k 2023-03-20T10:10:49.725 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:10:49.725 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-20T10:10:49.725 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-20T10:10:51.358 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-20T10:10:51.359 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-20T10:10:51.359 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-20T10:10:51.359 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-20T10:10:51.359 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-20T10:10:51.359 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:52.914 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:10:52.914 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================== 2023-03-20T10:10:52.914 INFO:teuthology.orchestra.run.smithi072.stdout:Install 13 Packages 2023-03-20T10:10:54.558 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 1.9 M 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 25 M 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 276 kB/s | 27 kB 00:00 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 159 kB/s | 20 kB 00:00 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.3 MB/s | 126 kB 00:00 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-03-20T10:10:54.559 INFO:teuthology.orchestra.run.smithi072.stdout:(6/13): ceph-mgr-rook-18.0.0-2743.ga6dd6683.el8 207 kB/s | 47 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 557 kB/s | 43 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 215 kB/s | 33 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 106 kB/s | 16 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 400 kB/s | 81 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 431 kB/s | 61 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 385 kB/s | 61 kB 00:00 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.0 MB/s | 1.2 MB 00:01 2023-03-20T10:10:54.560 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:10:54.561 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2023-03-20T10:10:54.561 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:10:55.847 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:10:55.847 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:10:57.075 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:10:57.075 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:10:57.075 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 13/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 13/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 1/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-03-20T10:10:57.077 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:57.078 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-20T10:10:58.176 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-20T10:10:58.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:10:58.177 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:10:58.177 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-cephadm 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-03-20T10:10:58.180 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 13/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 13/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 1/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-03-20T10:10:58.181 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-20T10:10:58.182 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-20T10:10:58.671 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-20T10:10:59.285 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:10:59.286 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:10:59.286 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-cephadm 2023-03-20T10:11:00.371 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:02 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:00.371 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:00.371 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:00.372 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:11:01.604 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 123 k 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-20T10:11:02.650 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:Install 5 Packages 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 6.0 M 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 26 M 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:11:02.651 INFO:teuthology.orchestra.run.smithi088.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.9 MB/s | 538 kB 00:00 2023-03-20T10:11:03.586 INFO:teuthology.orchestra.run.smithi088.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-03-20T10:11:03.586 INFO:teuthology.orchestra.run.smithi088.stdout:(3/5): ceph-mgr-cephadm-18.0.0-2743.ga6dd6683.e 571 kB/s | 123 kB 00:00 2023-03-20T10:11:03.589 INFO:teuthology.orchestra.run.smithi088.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 233 kB/s | 60 kB 00:00 2023-03-20T10:11:04.949 INFO:teuthology.orchestra.run.smithi088.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 827 kB/s | 476 kB 00:00 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Total 8.9 MB/s | 6.0 MB 00:00 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-03-20T10:11:04.950 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 5/5 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:03 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 123 k 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:11:04.952 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:Install 5 Packages 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 6.0 M 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 26 M 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.3 MB/s | 538 kB 00:00 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:(3/5): ceph-mgr-cephadm-18.0.0-2743.ga6dd6683.e 490 kB/s | 123 kB 00:00 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 249 kB/s | 60 kB 00:00 2023-03-20T10:11:04.953 INFO:teuthology.orchestra.run.smithi072.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 633 kB/s | 476 kB 00:00 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Total 7.0 MB/s | 6.0 MB 00:00 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-03-20T10:11:04.954 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-03-20T10:11:04.955 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 5/5 2023-03-20T10:11:06.282 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/5 2023-03-20T10:11:06.282 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-03-20T10:11:06.282 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-03-20T10:11:06.282 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-20T10:11:06.284 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-20T10:11:06.285 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-20T10:11:07.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:07.792 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:07.793 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-fuse 2023-03-20T10:11:07.794 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-03-20T10:11:07.794 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-03-20T10:11:10.040 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 5/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 5/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 1/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-20T10:11:10.041 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-20T10:11:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-20T10:11:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-20T10:11:10.584 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:10.585 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:10.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-fuse 2023-03-20T10:11:12.308 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:16 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:13.174 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:14 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:13.504 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:14.400 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:15.156 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 844 k 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Install 2 Packages 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 927 k 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 3.0 M 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:11:15.157 INFO:teuthology.orchestra.run.smithi088.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:(2/2): ceph-fuse-18.0.0-2743.ga6dd6683.el8.x86_ 3.1 MB/s | 844 kB 00:00 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Total 3.4 MB/s | 927 kB 00:00 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:11:15.158 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:15.692 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:11:15.693 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 844 k 2023-03-20T10:11:15.693 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:11:15.693 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-20T10:11:16.433 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:16.433 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:11:16.433 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 927 k 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 3.0 M 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): ceph-fuse-18.0.0-2743.ga6dd6683.el8.x86_ 2.9 MB/s | 844 kB 00:00 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Total 3.2 MB/s | 927 kB 00:00 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:11:16.434 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/2 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/2 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/2 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:11:16.436 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/2 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2/2 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/2 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:16.437 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:16.437 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-volume 2023-03-20T10:11:16.438 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-volume 2023-03-20T10:11:17.162 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:19 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:17.860 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:21 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:18.712 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:20.849 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:20.849 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2023-03-20T10:11:20.849 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 263 k 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:20.850 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:11:22.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:22.480 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 263 k 2023-03-20T10:11:22.480 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 1.3 M 2023-03-20T10:11:22.480 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:11:22.480 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-volume-18.0.0-2743.ga6dd6683.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:11:22.481 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:22.482 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:22.482 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:11:22.482 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:18.0.0-2743.ga6dd6683.el8 ceph-noarch 263 k 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 263 k 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 1.3 M 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:11:24.342 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-volume-18.0.0-2743.ga6dd6683.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:11:26.187 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:11:26.189 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:11:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 1/1 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:18.0.0-2743.ga6dd6683.el8.noarch 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:29.450 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:29.451 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados-devel 2023-03-20T10:11:29.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados-devel 2023-03-20T10:11:30.186 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:34 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:31.803 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:31.803 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:31.803 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2023-03-20T10:11:31.803 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 139 k 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:32.389 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:11:32.390 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:32.390 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 139 k 2023-03-20T10:11:32.390 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 512 k 2023-03-20T10:11:32.390 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:32 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:32.391 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 139 k 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:11:32.939 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 139 k 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 512 k 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:librados-devel-18.0.0-2743.ga6dd6683.el8.x86_64 739 kB/s | 139 kB 00:00 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Total 733 kB/s | 139 kB 00:00 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:11:32.940 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:11:34.811 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:11:34.812 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:11:34.812 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:11:34.812 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:34.812 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:34.813 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:36.577 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:36.577 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:11:36.577 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:11:36.577 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:36.577 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:36.578 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs2 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:librados-devel-18.0.0-2743.ga6dd6683.el8.x86_64 338 kB/s | 139 kB 00:00 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:Total 337 kB/s | 139 kB 00:00 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:11:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:11:36.580 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:11:36.580 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:11:36.580 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:39.318 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:39.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs2 2023-03-20T10:11:40.044 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:42 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:41.198 INFO:teuthology.orchestra.run.smithi072.stdout:Package libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:11:41.729 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:44 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:42.970 INFO:teuthology.orchestra.run.smithi088.stdout:Package libcephfs2-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:11:42.972 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:42.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs-devel 2023-03-20T10:11:43.315 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs-devel 2023-03-20T10:11:44.052 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:48 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:44.443 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:46 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:45.252 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:46.124 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:46.124 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2023-03-20T10:11:47.492 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:47.492 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:11:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 30 k 2023-03-20T10:11:47.493 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 30 k 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 137 k 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:libcephfs-devel-18.0.0-2743.ga6dd6683.el8.x86_6 201 kB/s | 30 kB 00:00 2023-03-20T10:11:49.647 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Total 199 kB/s | 30 kB 00:00 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:11:50.747 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 30 k 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 30 k 2023-03-20T10:11:50.748 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 137 k 2023-03-20T10:11:50.749 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:11:50.749 INFO:teuthology.orchestra.run.smithi072.stdout:libcephfs-devel-18.0.0-2743.ga6dd6683.el8.x86_6 217 kB/s | 30 kB 00:00 2023-03-20T10:11:50.749 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Total 215 kB/s | 30 kB 00:00 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:51.309 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:51.311 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:51.805 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:11:52.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:52.382 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:11:52.382 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:11:52.383 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:11:52.383 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:52.383 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados2 2023-03-20T10:11:52.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados2 2023-03-20T10:11:53.111 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:55 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:53.345 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:57 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:11:54.723 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:54.724 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librbd1 2023-03-20T10:11:54.725 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librbd1 2023-03-20T10:11:55.445 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:03:59 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:11:56.463 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:03:57 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:11:57.603 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:11:59.594 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:11:59.594 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:11:59.594 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:11:59.595 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:00.593 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:00.593 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:12:00.593 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:00.593 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rados 2023-03-20T10:12:00.594 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rados 2023-03-20T10:12:01.313 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:05 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:02.429 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:03.821 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:03 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:03.821 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rados-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:03.823 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:05.307 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:12:05.307 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:05.307 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:05.307 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:12:05.307 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:05.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rgw 2023-03-20T10:12:05.308 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rgw 2023-03-20T10:12:08.065 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:10 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:09.417 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:10.151 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:10.152 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:12:10.152 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:10.152 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:12 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:11.895 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rgw-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:11.896 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:11.896 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:12:11.896 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:13.096 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-cephfs 2023-03-20T10:12:13.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-cephfs 2023-03-20T10:12:13.824 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:15 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:16.043 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:17 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-cephfs-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:12:16.748 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:16.749 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rbd 2023-03-20T10:12:18.174 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rbd 2023-03-20T10:12:18.904 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:22 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:21.301 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:21.302 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:21.302 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2023-03-20T10:12:21.302 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:21.302 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:20 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:21.302 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 is already installed. 2023-03-20T10:12:22.086 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:22.086 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:12:22.086 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:22.086 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-fuse 2023-03-20T10:12:22.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-fuse 2023-03-20T10:12:22.811 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:26 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:22.911 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:24 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:24.013 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:24.970 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:24.970 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2023-03-20T10:12:24.970 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:24.970 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:12:24.970 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 94 k 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 94 k 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 280 k 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-fuse-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 528 kB/s | 94 kB 00:00 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Total 524 kB/s | 94 kB 00:00 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:12:24.971 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-03-20T10:12:24.972 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:25.927 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 94 k 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:25.928 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 94 k 2023-03-20T10:12:26.509 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 280 k 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-fuse-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 542 kB/s | 94 kB 00:00 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Total 538 kB/s | 94 kB 00:00 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:12:26.510 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:12:27.024 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:12:27.024 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:12:27.024 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:27.024 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:27.026 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:27.027 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:27.027 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:12:27.027 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:12:27.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:27.996 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:27.997 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-mirror 2023-03-20T10:12:27.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-mirror 2023-03-20T10:12:28.730 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:32 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:29.403 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:30 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:30.421 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:12:32.131 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:32.131 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2023-03-20T10:12:32.131 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.4 M 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 3.4 M 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 16 M 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-mirror-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 7.2 MB/s | 3.4 MB 00:00 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Total 7.2 MB/s | 3.4 MB 00:00 2023-03-20T10:12:32.132 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 3.4 M 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:12:32.134 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 3.4 M 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 16 M 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-mirror-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 8.1 MB/s | 3.4 MB 00:00 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Total 8.1 MB/s | 3.4 MB 00:00 2023-03-20T10:12:32.786 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:12:33.321 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:12:33.321 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:33.321 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:33.322 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:12:34.070 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:12:34.070 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-nbd 2023-03-20T10:12:42.638 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-nbd 2023-03-20T10:12:43.373 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:04:45 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:12:43.999 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:04:47 ago on Mon 20 Mar 2023 10:07:56 AM UTC. 2023-03-20T10:12:44.579 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2023-03-20T10:12:45.618 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 179 k 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 179 k 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 568 k 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-nbd-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 979 kB/s | 179 kB 00:00 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Total 972 kB/s | 179 kB 00:00 2023-03-20T10:12:45.619 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2023-03-20T10:12:45.620 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2023-03-20T10:12:45.620 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2023-03-20T10:12:45.620 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2023-03-20T10:13:07.394 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2023-03-20T10:13:10.419 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2023-03-20T10:13:10.419 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:10.419 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:10.419 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:13:11.241 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:13:11.241 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-03-20T10:13:11.241 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:13:11.242 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:13:11.242 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:18.0.0-2743.ga6dd6683.el8 ceph 179 k 2023-03-20T10:13:12.282 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 179 k 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 568 k 2023-03-20T10:13:17.794 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:13:19.233 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-nbd-18.0.0-2743.ga6dd6683.el8.x86_64.rpm 1.0 MB/s | 179 kB 00:00 2023-03-20T10:13:19.233 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:13:19.233 INFO:teuthology.orchestra.run.smithi072.stdout:Total 995 kB/s | 179 kB 00:00 2023-03-20T10:13:19.233 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:13:19.233 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:13:31.516 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:13:31.517 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:13:31.517 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:13:32.650 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:13:32.650 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:32.650 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:32.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 1/1 2023-03-20T10:13:32.652 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:13:38.997 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2023-03-20T10:13:38.997 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:18.0.0-2743.ga6dd6683.el8.x86_64 2023-03-20T10:13:41.356 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:13:41.356 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2023-03-20T10:13:41.357 DEBUG:teuthology.parallel:result is None 2023-03-20T10:13:41.357 DEBUG:teuthology.parallel:result is None 2023-03-20T10:13:41.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6dd66830c12498d02f3894e352dd8b49a7bab4c 2023-03-20T10:13:41.513 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-20T10:13:42.049 INFO:teuthology.orchestra.run.smithi072.stdout:18.0.0-2743.ga6dd6683.el8 2023-03-20T10:13:42.587 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2743.ga6dd6683.el8 2023-03-20T10:13:42.588 INFO:teuthology.task.install:The correct ceph version 18.0.0-2743.ga6dd6683 is installed. 2023-03-20T10:13:42.588 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6dd66830c12498d02f3894e352dd8b49a7bab4c 2023-03-20T10:13:44.562 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-20T10:13:46.523 INFO:teuthology.orchestra.run.smithi088.stdout:18.0.0-2743.ga6dd6683.el8 2023-03-20T10:13:48.762 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2743.ga6dd6683.el8 2023-03-20T10:13:48.762 INFO:teuthology.task.install:The correct ceph version 18.0.0-2743.ga6dd6683 is installed. 2023-03-20T10:13:48.762 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-20T10:13:48.762 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:13:48.763 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-20T10:13:48.793 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:13:50.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-20T10:13:50.386 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-20T10:13:52.188 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:13:52.188 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2023-03-20T10:13:52.221 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-20T10:13:53.357 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:13:54.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2023-03-20T10:13:55.012 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-20T10:13:55.720 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-20T10:13:56.330 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:13:56.330 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-20T10:13:56.361 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-20T10:13:58.620 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:13:59.980 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-20T10:14:00.009 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-20T10:14:01.864 INFO:teuthology.run_tasks:Running task ceph... 2023-03-20T10:14:10.996 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-20T10:14:19.204 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2023-03-20T10:14:21.142 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2023-03-20T10:14:21.173 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-20T10:14:23.131 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-20T10:14:23.134 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-20T10:14:24.783 INFO:tasks.ceph:Creating extra log directories... 2023-03-20T10:14:26.867 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-20T10:14:26.870 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-20T10:14:26.912 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-20T10:14:28.705 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-20T10:14:28.705 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi/7212642', 'branch': 'reef', 'description': 'rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}', 'email': None, 'first_in_suite': False, 'job_id': '7212642', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-03-17_23:46:23-rgw-reef-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c', 'wait-for-scrub': False}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c'}}, 'rgw': {'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_a6dd66830c12498d02f3894e352dd8b49a7bab4c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'a6dd66830c12498d02f3894e352dd8b49a7bab4c', 'targets': {'smithi072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHQ4yea4haxdr0Tc1sKDpUA30E26tyZ5GHPbuAYKSkdjA+Xn6I76KU6CIrexLrMe8XM/8H2FjJETZjXoFBoIke8=', 'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC74ZJP/jMnXnRPjq0oFFcua5P/jsurKd9v8GUcROUrbaHWZE5e2Y38QftDbrLEzt0tAvFpIUEMvvJDd/BYJbQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'rabbitmq': {'client.0': None}}, {'notification-tests': {'client.0': {'extra_attr': ['amqp_test'], 'rgw_server': 'client.0'}}}, {'kafka': {'client.0': {'kafka_version': '2.6.0'}}}, {'notification-tests': {'client.0': {'extra_attr': ['kafka_test'], 'rgw_server': 'client.0'}}}, {'notification-tests': {'client.0': {'rgw_server': 'client.0'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '7f7f7c33a372789d0252bbbcd5798300b0f044d0', 'timestamp': '2023-03-17_23:46:23', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.4737'} 2023-03-20T10:14:28.706 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-20T10:14:28.708 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-20T10:14:31.125 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2023-03-20T10:14:32.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2023-03-20T10:14:34.653 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:14:36.386 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2023-03-20T10:14:36.406 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-20T10:14:38.547 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2023-03-20T10:14:38.567 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 371384 Links: 1 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-20 10:12:31.316276282 +0000 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-20 10:06:03.089819779 +0000 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-20 10:06:03.089819779 +0000 2023-03-20T10:14:40.422 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-20T10:14:40.422 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-20T10:14:40.453 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-20T10:14:41.625 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-20T10:14:41.625 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000271186 s, 1.9 MB/s 2023-03-20T10:14:41.626 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-20T10:14:41.687 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2023-03-20T10:14:44.745 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-20T10:14:46.024 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:14:46.024 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 378722 Links: 1 2023-03-20T10:14:46.024 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:14:46.024 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:14:46.025 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-20 10:12:31.316276282 +0000 2023-03-20T10:14:46.025 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-20 10:06:03.589809168 +0000 2023-03-20T10:14:46.025 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-20 10:06:03.589809168 +0000 2023-03-20T10:14:46.025 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-20T10:14:46.025 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-20T10:14:46.056 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-20T10:14:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-20T10:14:47.082 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000236594 s, 2.2 MB/s 2023-03-20T10:14:47.084 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-20T10:14:48.128 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2023-03-20T10:14:49.508 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 385185 Links: 1 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-20 10:12:31.316276282 +0000 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-20 10:06:04.104798239 +0000 2023-03-20T10:14:49.885 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-20 10:06:04.104798239 +0000 2023-03-20T10:14:49.886 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-20T10:14:49.886 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-20T10:14:50.307 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-20T10:14:52.438 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-20T10:14:53.056 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000251614 s, 2.0 MB/s 2023-03-20T10:14:53.058 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-20T10:14:53.119 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2023-03-20T10:14:54.097 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-20T10:14:55.301 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:14:55.301 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 6h/6d Inode: 381804 Links: 1 2023-03-20T10:14:55.301 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:14:55.908 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:14:55.908 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2023-03-20 10:12:31.316276282 +0000 2023-03-20T10:14:55.908 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2023-03-20 10:06:04.625787182 +0000 2023-03-20T10:14:55.909 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2023-03-20 10:06:04.625787182 +0000 2023-03-20T10:14:55.909 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: - 2023-03-20T10:14:55.909 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-20T10:14:55.938 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2023-03-20T10:14:56.496 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2023-03-20T10:14:56.496 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000178864 s, 2.9 MB/s 2023-03-20T10:14:56.497 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-20T10:14:57.289 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-20T10:14:58.826 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:14:58.827 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2023-03-20T10:15:00.342 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-20T10:15:01.280 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2023-03-20T10:15:03.477 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-20T10:15:06.574 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 386056 Links: 1 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-03-20 10:12:31.276374027 +0000 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-03-20 10:05:56.185891084 +0000 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-03-20 10:05:56.185891084 +0000 2023-03-20T10:15:06.575 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2023-03-20T10:15:06.575 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-20T10:15:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-03-20T10:15:09.771 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-03-20T10:15:09.771 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000258727 s, 2.0 MB/s 2023-03-20T10:15:09.772 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-20T10:15:09.832 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2023-03-20T10:15:11.422 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-20T10:15:20.537 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 384465 Links: 1 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-03-20 10:12:31.276374027 +0000 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-03-20 10:05:56.703880562 +0000 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-03-20 10:05:56.703880562 +0000 2023-03-20T10:15:20.538 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2023-03-20T10:15:20.538 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-20T10:15:20.567 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-03-20T10:15:22.518 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-03-20T10:15:22.519 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000221361 s, 2.3 MB/s 2023-03-20T10:15:22.520 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-20T10:15:22.948 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2023-03-20T10:15:24.624 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-20T10:15:35.240 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:15:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 384517 Links: 1 2023-03-20T10:15:35.240 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:15:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:15:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-03-20 10:12:31.276374027 +0000 2023-03-20T10:15:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-03-20 10:05:57.200870467 +0000 2023-03-20T10:15:35.241 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-03-20 10:05:57.200870467 +0000 2023-03-20T10:15:35.241 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2023-03-20T10:15:35.242 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-20T10:15:35.269 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-03-20T10:15:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-03-20T10:15:36.678 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000215725 s, 2.4 MB/s 2023-03-20T10:15:36.679 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-20T10:15:36.742 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2023-03-20T10:15:38.122 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 385642 Links: 1 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2023-03-20 10:12:31.276374027 +0000 2023-03-20T10:15:40.222 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2023-03-20 10:05:57.705860208 +0000 2023-03-20T10:15:42.976 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2023-03-20 10:05:57.705860208 +0000 2023-03-20T10:15:42.976 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2023-03-20T10:15:42.977 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-20T10:15:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2023-03-20T10:15:47.073 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2023-03-20T10:15:47.073 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000242628 s, 2.1 MB/s 2023-03-20T10:15:47.075 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-20T10:15:47.136 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-20T10:15:50.370 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-20T10:15:50.370 INFO:tasks.ceph:Generating config... 2023-03-20T10:15:50.372 INFO:tasks.ceph:[client] debug rgw = 20 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] setgroup = ceph 2023-03-20T10:15:52.328 INFO:tasks.ceph:[client] setuser = ceph 2023-03-20T10:15:52.328 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2023-03-20T10:15:52.328 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2023-03-20T10:15:52.328 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-20T10:15:52.328 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-20T10:15:52.328 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-20T10:15:52.328 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-20T10:15:52.328 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-20T10:15:52.329 INFO:tasks.ceph:[osd] bdev async discard = True 2023-03-20T10:15:52.329 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-03-20T10:15:52.329 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-03-20T10:15:52.330 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-20T10:15:52.330 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-20T10:15:52.331 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-03-20T10:15:52.331 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-03-20T10:15:54.370 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-20T10:15:54.370 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-20T10:15:54.370 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-03-20T10:15:54.370 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-20T10:15:54.370 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-20T10:15:54.371 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-20T10:15:56.207 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-20T10:15:56.207 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-20T10:15:56.207 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-03-20T10:15:56.207 INFO:tasks.ceph:Setting up mon.a... 2023-03-20T10:15:56.207 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-20T10:15:56.266 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.keyring 2023-03-20T10:16:00.913 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-20T10:16:00.966 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-20T10:16:04.018 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.72'), ('mon.c', '[v2:172.21.15.72:3301,v1:172.21.15.72:6790]'), ('mon.b', '172.21.15.88')] 2023-03-20T10:16:05.949 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790],172.21.15.88', 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-03-20T10:16:05.950 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:16:05.950 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-20T10:16:05.970 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.72 --addv c '[v2:172.21.15.72:3301,v1:172.21.15.72:6790]' --add b 172.21.15.88 --print /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:16:07.257 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:16:08.687 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:16:08.689 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:16:08.689 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: generated fsid 608e0299-7590-4702-bfbc-29e629ec4b60 2023-03-20T10:16:09.039 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = pacific 2023-03-20T10:16:09.039 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 0 2023-03-20T10:16:09.579 INFO:teuthology.orchestra.run.smithi072.stdout:fsid 608e0299-7590-4702-bfbc-29e629ec4b60 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:last_changed 2023-03-20T10:16:07.258508+0000 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:created 2023-03-20T10:16:07.258508+0000 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:min_mon_release 16 (pacific) 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:election_strategy: 1 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:0: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.a 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:1: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2023-03-20T10:16:09.580 INFO:teuthology.orchestra.run.smithi072.stdout:2: [v2:172.21.15.72:3301/0,v1:172.21.15.72:6790/0] mon.c 2023-03-20T10:16:10.581 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-20T10:16:10.581 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-20T10:16:11.255 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 608e0299-7590-4702-bfbc-29e629ec4b60... 2023-03-20T10:16:12.547 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-20T10:16:12.550 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-20T10:16:13.762 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2023-03-20T10:16:13.762 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2023-03-20T10:16:14.086 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-20T10:16:14.086 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2023-03-20T10:16:14.086 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.086 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2023-03-20T10:16:14.637 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.637 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2023-03-20T10:16:14.984 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2023-03-20T10:16:14.985 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2023-03-20T10:16:14.986 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2023-03-20T10:16:14.986 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-20T10:16:15.311 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-20T10:16:15.639 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2023-03-20T10:16:15.639 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:15.639 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-20T10:16:15.639 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:15.833 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2023-03-20T10:16:15.833 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:15.833 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2023-03-20T10:16:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2023-03-20T10:16:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2023-03-20T10:16:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2023-03-20T10:16:16.163 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790],172.21.15.88" 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd_max_pg_log_entries = 10 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd_min_pg_log_entries = 10 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = 608e0299-7590-4702-bfbc-29e629ec4b60 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:16.993 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.199 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bdev async discard = True 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bdev enable discard = True 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore allocator = bitmap 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 1/20 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 1/20 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 4/10 2023-03-20T10:16:17.200 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: osd shutdown pgref assert = True 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2023-03-20T10:16:17.201 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2023-03-20T10:16:17.202 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2023-03-20T10:16:17.865 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2023-03-20T10:16:17.865 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2023-03-20T10:16:17.865 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2023-03-20T10:16:17.865 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:17.865 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-20T10:16:18.469 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2023-03-20T10:16:18.470 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2023-03-20T10:16:18.470 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:24.181 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-20T10:16:24.182 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-20T10:16:24.183 INFO:teuthology.orchestra.run.smithi072.stdout: debug rgw = 20 2023-03-20T10:16:24.183 INFO:teuthology.orchestra.run.smithi072.stdout: rgw crypt require ssl = False 2023-03-20T10:16:24.183 INFO:teuthology.orchestra.run.smithi072.stdout: rgw crypt s3 kms backend = testing 2023-03-20T10:16:25.264 INFO:teuthology.orchestra.run.smithi072.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout: rgw lc debug interval = 10 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout: setgroup = ceph 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout: setuser = ceph 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2023-03-20T10:16:25.265 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2023-03-20T10:16:25.267 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2023-03-20T10:16:25.267 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2023-03-20T10:16:25.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:25.269 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2023-03-20T10:16:25.270 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2023-03-20T10:16:25.270 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2023-03-20T10:16:26.581 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2023-03-20T10:16:26.581 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2023-03-20T10:16:26.581 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2023-03-20T10:16:26.581 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2023-03-20T10:16:26.582 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-20T10:16:26.582 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.854 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.72,[v2:172.21.15.72:3301,v1:172.21.15.72:6790],172.21.15.88" 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd_max_pg_log_entries = 10 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd_min_pg_log_entries = 10 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = 608e0299-7590-4702-bfbc-29e629ec4b60 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2023-03-20T10:16:28.855 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2023-03-20T10:16:28.856 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2023-03-20T10:16:28.857 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2023-03-20T10:16:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2023-03-20T10:16:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: osd shutdown pgref assert = True 2023-03-20T10:16:29.942 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:29.943 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2023-03-20T10:16:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-03-20T10:16:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2023-03-20T10:16:29.943 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2023-03-20T10:16:29.944 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:29.945 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-20T10:16:29.945 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2023-03-20T10:16:31.796 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2023-03-20T10:16:31.797 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2023-03-20T10:16:33.914 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: debug rgw = 20 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt require ssl = False 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt s3 kms backend = testing 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: rgw lc debug interval = 10 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: setgroup = ceph 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: setuser = ceph 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2023-03-20T10:16:34.834 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2023-03-20T10:16:34.835 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-20T10:16:34.835 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-20T10:16:34.895 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-20T10:16:37.677 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:16:37.677 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-20T10:16:37.696 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:16:39.663 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-20T10:16:39.683 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T10:16:40.995 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:16:40.995 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-20T10:16:40.995 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-20T10:16:41.037 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:16:42.177 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:16:43.011 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T10:16:44.641 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:16:44.641 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-20T10:16:44.641 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-20T10:16:44.683 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:16:45.261 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:16:45.281 INFO:tasks.ceph:Setting up mon nodes... 2023-03-20T10:16:46.413 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-20T10:16:46.413 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-20T10:16:46.474 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-20T10:16:47.442 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-20T10:16:47.529 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-20T10:16:49.186 INFO:tasks.ceph:Setting up mds nodes... 2023-03-20T10:16:49.187 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-20T10:16:49.187 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-20T10:16:49.234 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-20T10:16:49.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-20T10:16:50.017 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-20T10:16:51.796 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-20T10:16:51.797 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-20T10:16:51.797 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-20T10:16:52.467 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-20T10:16:53.982 INFO:tasks.ceph:role: osd.0 2023-03-20T10:16:55.772 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T10:16:55.772 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-20T10:16:55.809 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:17:08.280 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:17:08.282 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-20T10:17:08.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-20T10:17:09.701 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-20T10:17:09.750 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-20T10:17:12.861 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-20T10:17:26.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-20T10:17:28.888 INFO:tasks.ceph:role: osd.1 2023-03-20T10:17:28.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T10:17:28.888 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-20T10:17:28.926 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:17:31.308 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:17:31.308 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:17:31.309 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:17:34.102 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:17:34.103 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:17:34.103 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:17:34.103 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:17:34.103 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:17:34.103 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:17:34.104 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-20T10:17:34.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-20T10:17:34.105 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-20T10:17:34.192 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-20T10:17:36.571 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-20T10:17:39.783 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-20T10:17:42.161 INFO:tasks.ceph:role: osd.2 2023-03-20T10:17:42.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T10:17:42.161 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-20T10:17:42.750 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:17:43.921 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:17:43.922 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:17:43.922 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:17:43.922 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:17:43.923 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-20T10:17:43.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-20T10:17:43.923 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-20T10:17:44.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-20T10:17:45.132 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-20T10:17:46.411 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-20T10:17:47.377 INFO:tasks.ceph:role: osd.3 2023-03-20T10:17:47.377 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com 2023-03-20T10:17:47.377 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-20T10:17:47.416 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:17:48.581 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:17:49.945 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:17:49.946 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2023-03-20T10:17:51.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2023-03-20T10:17:51.854 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-20T10:17:51.903 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-20T10:17:53.978 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:17:56.000 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.000+0000 7fd0bd7e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-20T10:18:01.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.000+0000 7fd0bd7e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-20T10:18:01.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.000+0000 7fd0bd7e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-20T10:18:01.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.000+0000 7fd0bd7e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-20T10:18:01.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.000+0000 7fd0bd7e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-20T10:18:01.070 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:17:56.001+0000 7fd0bd7e3640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-20T10:18:01.070 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-20T10:18:01.100 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:18:03.989 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.989+0000 7faae3b5a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-20T10:18:04.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.989+0000 7faae3b5a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-20T10:18:04.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.989+0000 7faae3b5a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-20T10:18:04.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.989+0000 7faae3b5a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-20T10:18:04.929 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.989+0000 7faae3b5a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-20T10:18:04.930 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:03.990+0000 7faae3b5a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-20T10:18:06.834 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-20T10:18:07.766 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:18:08.841 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.840+0000 7fb03c513640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-20T10:18:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.841+0000 7fb03c513640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-20T10:18:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.841+0000 7fb03c513640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-20T10:18:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.841+0000 7fb03c513640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-20T10:18:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.841+0000 7fb03c513640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-20T10:18:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:08.841+0000 7fb03c513640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-20T10:18:11.682 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-20T10:18:12.675 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:18:13.409 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.408+0000 7fe59b6b9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-20T10:18:14.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.408+0000 7fe59b6b9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-20T10:18:14.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.408+0000 7fe59b6b9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-20T10:18:14.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.408+0000 7fe59b6b9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-20T10:18:14.325 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.408+0000 7fe59b6b9640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-20T10:18:14.327 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:18:13.409+0000 7fe59b6b9640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-20T10:18:16.259 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-20T10:18:36.909 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-20T10:18:38.330 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-20T10:18:39.909 INFO:tasks.ceph:role: osd.4 2023-03-20T10:18:39.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T10:18:39.909 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-20T10:18:39.946 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:18:41.068 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:18:41.068 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:18:41.068 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:18:41.068 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:18:41.069 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:18:41.069 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:18:41.069 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:18:41.069 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:18:41.069 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:18:41.070 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-03-20T10:18:41.070 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-03-20T10:18:41.070 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-20T10:18:41.159 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-20T10:18:52.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-20T10:18:54.287 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-20T10:18:56.822 INFO:tasks.ceph:role: osd.5 2023-03-20T10:18:56.822 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T10:18:56.823 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-20T10:18:56.860 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:18:58.583 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:18:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:18:58.586 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-03-20T10:18:59.404 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-03-20T10:19:08.437 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-20T10:19:08.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-20T10:19:10.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-20T10:19:13.075 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-20T10:19:15.170 INFO:tasks.ceph:role: osd.6 2023-03-20T10:19:15.170 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T10:19:15.170 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-20T10:19:15.826 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:19:16.579 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:19:16.580 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:19:16.581 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-03-20T10:19:16.582 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-03-20T10:19:16.582 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-20T10:19:16.761 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-20T10:19:17.570 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-20T10:19:17.911 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-20T10:19:18.225 INFO:tasks.ceph:role: osd.7 2023-03-20T10:19:18.225 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2023-03-20T10:19:18.226 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-20T10:19:18.263 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-20T10:19:20.332 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-20T10:19:22.162 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-20T10:19:22.162 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-20T10:19:22.773 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-20T10:19:22.774 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2023-03-20T10:19:22.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2023-03-20T10:19:22.775 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-20T10:19:22.868 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-20T10:19:25.090 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:19:25.681 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.680+0000 7f0c279dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-20T10:19:26.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.680+0000 7f0c279dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-20T10:19:26.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.680+0000 7f0c279dc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-20T10:19:26.264 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.680+0000 7f0c279dc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-20T10:19:26.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.680+0000 7f0c279dc640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-20T10:19:26.859 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:25.681+0000 7f0c279dc640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-20T10:19:28.528 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-20T10:19:30.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:19:31.631 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-20T10:19:42.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-20T10:19:42.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-20T10:19:42.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-20T10:19:42.622 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-20T10:19:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:31.631+0000 7f05f3189640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-20T10:19:42.624 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-20T10:19:42.655 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:19:43.910 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.909+0000 7fb8d7d10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-20T10:19:45.227 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.910+0000 7fb8d7d10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-20T10:19:45.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.910+0000 7fb8d7d10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-20T10:19:45.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.910+0000 7fb8d7d10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-20T10:19:45.228 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.910+0000 7fb8d7d10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-20T10:19:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:43.910+0000 7fb8d7d10640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-20T10:19:46.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-20T10:19:48.212 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:19:49.249 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.248+0000 7fe941e77640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-20T10:19:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.248+0000 7fe941e77640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-20T10:19:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.248+0000 7fe941e77640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-20T10:19:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.248+0000 7fe941e77640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-20T10:19:50.173 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.248+0000 7fe941e77640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-20T10:19:50.174 INFO:teuthology.orchestra.run.smithi088.stderr:2023-03-20T10:19:49.249+0000 7fe941e77640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-20T10:19:52.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-20T10:19:56.784 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-20T10:19:57.714 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:19:58.533 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-20T10:19:58.562 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:19:59.185 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-20T10:19:59.213 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:20:00.639 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-20T10:20:00.669 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:20:01.849 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-20T10:20:01.882 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:20:06.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-20T10:20:06.457 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:07.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-20T10:20:07.437 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:09.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-20T10:20:09.792 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:12.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-20T10:20:12.942 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:28.145 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-20T10:20:28.923 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:30.213 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-20T10:20:30.246 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:20:30.873 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-20T10:20:30.892 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:20:32.872 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-20T10:20:33.686 INFO:tasks.ceph:Adding keys to all mons... 2023-03-20T10:20:34.652 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-20T10:20:35.530 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-20T10:20:35.554 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.y] 2023-03-20T10:20:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCOMhhkHMZaHBAAnlN6cv/hCGMUrJhNOXZRVQ== 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDUMhhkc9cPABAAqillh61FsFVn4y8cKB7o4A== 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDbMhhkog8DOxAAxXw27mVX4S9XopY8XJCOWw== 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDgMhhksfgmMhAA+4LcjQmmK+FFzLtjb1/Uqg== 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.3] 2023-03-20T10:20:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQDlMhhkpqNyGBAAuN5z+nQsa5vytTTmXFScag== 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCPMhhkI0igHxAA49FW0BVaOWUCS20FsGDkTg== 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.4] 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAtMxhkZ3yiKBAAHgyo08soZfj3LBfb8d/VFg== 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.5] 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAzMxhkpf+wJRAA4+qatLs26B2j7K1XBwAzjw== 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.6] 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQA/Mxhk+VRQNhAAgnII1lGlOYiOedft+cnjMg== 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.7] 2023-03-20T10:20:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQBFMxhks6fmDhAAWZsHR0ZZrGk+/Nof9pK2hg== 2023-03-20T10:20:36.539 INFO:teuthology.orchestra.run.smithi072.stdout:[client.0] 2023-03-20T10:20:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCRMhhkWCT9DRAAvP6ifVXbwespDqow2xz9Zg== 2023-03-20T10:20:36.539 INFO:teuthology.orchestra.run.smithi072.stdout:[client.1] 2023-03-20T10:20:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQCSMhhk+QMfARAALThpwOinmjCxw98bChC3ew== 2023-03-20T10:20:36.541 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2023-03-20T10:20:36.541 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCOMhhkHMZaHBAAnlN6cv/hCGMUrJhNOXZRVQ== 2023-03-20T10:20:36.541 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2023-03-20T10:20:36.541 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDUMhhkc9cPABAAqillh61FsFVn4y8cKB7o4A== 2023-03-20T10:20:36.541 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2023-03-20T10:20:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDbMhhkog8DOxAAxXw27mVX4S9XopY8XJCOWw== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDgMhhksfgmMhAA+4LcjQmmK+FFzLtjb1/Uqg== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQDlMhhkpqNyGBAAuN5z+nQsa5vytTTmXFScag== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCPMhhkI0igHxAA49FW0BVaOWUCS20FsGDkTg== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAtMxhkZ3yiKBAAHgyo08soZfj3LBfb8d/VFg== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQAzMxhkpf+wJRAA4+qatLs26B2j7K1XBwAzjw== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQA/Mxhk+VRQNhAAgnII1lGlOYiOedft+cnjMg== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQBFMxhks6fmDhAAWZsHR0ZZrGk+/Nof9pK2hg== 2023-03-20T10:20:38.524 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2023-03-20T10:20:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCRMhhkWCT9DRAAvP6ifVXbwespDqow2xz9Zg== 2023-03-20T10:20:38.525 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2023-03-20T10:20:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCSMhhk+QMfARAALThpwOinmjCxw98bChC3ew== 2023-03-20T10:20:38.525 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-20T10:20:38.528 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-20T10:20:38.589 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:40.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:47.370 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:49.283 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:49.342 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:51.830 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:51.885 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:54.706 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:54.764 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-20T10:20:56.552 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-20T10:20:56.609 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:20:58.284 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:13.710 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:15.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:15.119 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:16.005 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:17.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:20.116 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-20T10:21:20.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-20T10:21:22.765 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-20T10:21:22.829 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-20T10:21:25.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-20T10:21:31.501 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-20T10:21:33.165 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-20T10:21:33.199 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-20T10:21:35.642 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-20T10:21:43.886 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-20T10:21:45.501 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-20T10:21:48.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-20T10:21:50.318 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-20T10:21:58.685 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-20T10:22:03.944 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-20T10:22:07.382 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:22:13.017 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-20T10:22:15.353 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-20T10:22:20.181 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-20T10:22:21.519 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-20T10:22:21.521 INFO:tasks.ceph.mon.a:Started 2023-03-20T10:22:23.407 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-20T10:22:23.407 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-20T10:22:23.410 INFO:tasks.ceph.mon.c:Started 2023-03-20T10:22:24.187 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-20T10:22:25.528 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-20T10:22:27.277 INFO:tasks.ceph.mon.b:Started 2023-03-20T10:22:27.277 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-20T10:22:27.277 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-20T10:22:28.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-20T10:22:30.600 INFO:tasks.ceph.mgr.y:Started 2023-03-20T10:22:30.600 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-20T10:22:30.600 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-20T10:22:30.603 INFO:tasks.ceph.mgr.x:Started 2023-03-20T10:22:32.117 INFO:tasks.ceph:Setting crush tunables to default 2023-03-20T10:22:32.117 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd crush tunables default 2023-03-20T10:22:32.990 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2023-03-20T10:22:34.137 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:33.812+0000 7f0311847200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-20T10:22:35.849 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:33.882+0000 7f0311847200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-20T10:22:35.849 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:33.964+0000 7f0311847200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-20T10:22:35.850 INFO:tasks.ceph:check_enable_crimson: False 2023-03-20T10:22:35.850 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-20T10:22:35.850 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:22:35.850 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-20T10:22:35.851 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:33.835+0000 7fcb077d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:33.906+0000 7fcb077d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:33.987+0000 7fcb077d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:34.171+0000 7fcb077d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:34.941+0000 7fcb077d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:35.081+0000 7fcb077d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:35.297+0000 7fcb077d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:35.368+0000 7fcb077d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-20T10:22:36.953 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:35.504+0000 7fcb077d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-20T10:22:36.955 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.038+0000 7fcb077d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-20T10:22:37.641 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.112+0000 7fcb077d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-20T10:22:37.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.250+0000 7fcb077d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-20T10:22:37.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.496+0000 7fcb077d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-20T10:22:37.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.637+0000 7fcb077d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-20T10:22:37.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.716+0000 7fcb077d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-20T10:22:37.642 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.795+0000 7fcb077d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-20T10:22:37.646 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:36.873+0000 7fcb077d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-20T10:22:37.647 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:34.149+0000 7f0311847200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:34.922+0000 7f0311847200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:35.063+0000 7f0311847200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:35.281+0000 7f0311847200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:35.353+0000 7f0311847200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:35.489+0000 7f0311847200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-20T10:22:38.417 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.025+0000 7f0311847200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.101+0000 7f0311847200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.242+0000 7f0311847200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.508+0000 7f0311847200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.649+0000 7f0311847200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.728+0000 7f0311847200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-20T10:22:38.418 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.809+0000 7f0311847200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-20T10:22:38.972 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:36.888+0000 7f0311847200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-20T10:22:39.181 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:22:39.181 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:37.652+0000 7fcb077d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:37.734+0000 7fcb077d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:37.808+0000 7fcb077d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:37.889+0000 7fcb077d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:38.071+0000 7fcb077d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:38.143+0000 7fcb077d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-20T10:22:39.183 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:38.502+0000 7fcb077d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-20T10:22:39.184 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:37.681+0000 7f0311847200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-20T10:22:39.184 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:37.764+0000 7f0311847200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-20T10:22:39.184 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:37.841+0000 7f0311847200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-20T10:22:39.479 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:37.924+0000 7f0311847200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-20T10:22:39.907 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:38.107+0000 7f0311847200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-20T10:22:41.128 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:38.180+0000 7f0311847200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-20T10:22:41.584 INFO:tasks.ceph.mgr.x.smithi088.stderr:2023-03-20T10:22:38.543+0000 7f0311847200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-20T10:22:41.584 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:39.524+0000 7fcac12e9700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-20T10:22:41.585 INFO:tasks.ceph.mgr.y.smithi072.stderr:2023-03-20T10:22:39.528+0000 7fcac12e9700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-20T10:22:41.616 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:22:42.407 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-20T10:22:42.438 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:22:43.472 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-20T10:22:43.508 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:22:43.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-20T10:22:44.535 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:22:45.172 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-20T10:22:45.204 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:22:45.803 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-20T10:22:45.836 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2023-03-20T10:22:46.700 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-20T10:22:46.734 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 7e2f177f-6f2a-4209-b131-5b35e996adc8 0 2023-03-20T10:22:48.203 INFO:teuthology.orchestra.run.smithi088.stdout:0 2023-03-20T10:22:49.502 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 93198d8a-c6c0-4743-a9a6-17e6cb4d57c1 1 2023-03-20T10:22:49.835 INFO:teuthology.orchestra.run.smithi088.stdout:1 2023-03-20T10:22:51.283 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new e38724c9-c316-48f0-8e50-3bb85e8c2eee 2 2023-03-20T10:22:51.617 INFO:teuthology.orchestra.run.smithi088.stdout:2 2023-03-20T10:22:53.261 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new fdbc2572-b14f-4014-a565-8563a0f026c1 3 2023-03-20T10:22:53.597 INFO:teuthology.orchestra.run.smithi088.stdout:3 2023-03-20T10:22:55.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 3b007991-2c19-45a3-9cc8-12e85291ba9b 4 2023-03-20T10:22:55.898 INFO:teuthology.orchestra.run.smithi088.stdout:4 2023-03-20T10:22:58.689 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 4feca5d3-6c51-47c2-a153-19d8d20e7047 5 2023-03-20T10:22:59.031 INFO:teuthology.orchestra.run.smithi088.stdout:5 2023-03-20T10:23:00.419 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 5533656a-12bf-43ae-bfac-896b15840290 6 2023-03-20T10:23:00.762 INFO:teuthology.orchestra.run.smithi088.stdout:6 2023-03-20T10:23:01.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd new 63b713e4-8511-4221-a5c1-1783e2b2b41c 7 2023-03-20T10:23:02.107 INFO:teuthology.orchestra.run.smithi088.stdout:7 2023-03-20T10:23:03.828 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-20T10:23:03.828 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-20T10:23:03.832 INFO:tasks.ceph.osd.0:Started 2023-03-20T10:23:05.525 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-20T10:23:07.017 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-20T10:23:07.976 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-20T10:23:04.431+0000 7fb0ac80e640 -1 Falling back to public interface 2023-03-20T10:23:07.976 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-20T10:23:07.851+0000 7fb0ac80e640 -1 osd.0 0 log_to_monitors true 2023-03-20T10:23:07.978 INFO:tasks.ceph.osd.1:Started 2023-03-20T10:23:09.027 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-20T10:23:09.027 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-20T10:23:09.028 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-20T10:23:08.572+0000 7f62c1891640 -1 Falling back to public interface 2023-03-20T10:23:09.029 INFO:tasks.ceph.osd.2:Started 2023-03-20T10:23:09.030 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-20T10:23:09.030 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-20T10:23:09.033 INFO:tasks.ceph.osd.3:Started 2023-03-20T10:23:10.515 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-20T10:23:10.515 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-20T10:23:10.516 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-20T10:23:09.624+0000 7f2dccfee640 -1 Falling back to public interface 2023-03-20T10:23:10.517 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-20T10:23:09.625+0000 7feb59824640 -1 Falling back to public interface 2023-03-20T10:23:10.517 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-20T10:23:10.490+0000 7fb0a03b4700 -1 osd.0 0 waiting for initial osdmap 2023-03-20T10:23:10.517 INFO:tasks.ceph.osd.0.smithi072.stderr:2023-03-20T10:23:10.505+0000 7fb09878f700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:10.518 INFO:tasks.ceph.osd.4:Started 2023-03-20T10:23:10.519 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-20T10:23:10.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-20T10:23:10.521 INFO:tasks.ceph.osd.5:Started 2023-03-20T10:23:10.521 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-20T10:23:10.521 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-20T10:23:10.524 INFO:tasks.ceph.osd.6:Started 2023-03-20T10:23:12.318 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-20T10:23:12.318 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-20T10:23:12.319 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-20T10:23:12.079+0000 7f62c1891640 -1 osd.1 0 log_to_monitors true 2023-03-20T10:23:12.319 INFO:tasks.ceph.osd.4.smithi088.stderr:2023-03-20T10:23:11.121+0000 7eff14993640 -1 Falling back to public interface 2023-03-20T10:23:12.320 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:11.121+0000 7f6ea3d39640 -1 Falling back to public interface 2023-03-20T10:23:12.320 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:11.121+0000 7fe12834b640 -1 Falling back to public interface 2023-03-20T10:23:12.321 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:11.121+0000 7f6e9b0e6700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-03-20T10:23:12.321 INFO:tasks.ceph.osd.4.smithi088.stderr:2023-03-20T10:23:11.121+0000 7eff0bd40700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-03-20T10:23:12.321 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:11.121+0000 7fe11f6f8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-03-20T10:23:12.322 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:11.121+0000 7fe11f6f8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2023-03-20T10:23:12.322 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:11.123+0000 7f6e9b0e6700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6819/0: (98) Address already in use 2023-03-20T10:23:12.325 INFO:tasks.ceph.osd.7:Started 2023-03-20T10:23:12.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-20T10:23:13.796 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-20T10:23:13.048+0000 7f2dccfee640 -1 osd.3 0 log_to_monitors true 2023-03-20T10:23:13.796 INFO:tasks.ceph.osd.7.smithi088.stderr:2023-03-20T10:23:12.926+0000 7fdaf1aa0640 -1 Falling back to public interface 2023-03-20T10:23:13.796 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-20T10:23:13.104+0000 7feb59824640 -1 osd.2 0 log_to_monitors true 2023-03-20T10:23:14.147 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:23:14.946 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":15,"fsid":"608e0299-7590-4702-bfbc-29e629ec4b60","created":"2023-03-20T10:22:32.881129+0000","modified":"2023-03-20T10:23:13.648987+0000","last_up_change":"2023-03-20T10:23:11.493324+0000","last_in_change":"2023-03-20T10:23:02.093998+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7e2f177f-6f2a-4209-b131-5b35e996adc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":110509},{"type":"v1","addr":"172.21.15.72:6803","nonce":110509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":110509},{"type":"v1","addr":"172.21.15.72:6805","nonce":110509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":110509},{"type":"v1","addr":"172.21.15.72:6809","nonce":110509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":110509},{"type":"v1","addr":"172.21.15.72:6807","nonce":110509}]},"public_addr":"172.21.15.72:6803/110509","cluster_addr":"172.21.15.72:6805/110509","heartbeat_back_addr":"172.21.15.72:6809/110509","heartbeat_front_addr":"172.21.15.72:6807/110509","state":["exists","up"]},{"osd":1,"uuid":"93198d8a-c6c0-4743-a9a6-17e6cb4d57c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e38724c9-c316-48f0-8e50-3bb85e8c2eee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fdbc2572-b14f-4014-a565-8563a0f026c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3b007991-2c19-45a3-9cc8-12e85291ba9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4feca5d3-6c51-47c2-a153-19d8d20e7047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5533656a-12bf-43ae-bfac-896b15840290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"63b713e4-8511-4221-a5c1-1783e2b2b41c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:08.835491+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-20T10:23:14.947 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:14.611+0000 7f6ea3d39640 -1 osd.5 0 log_to_monitors true 2023-03-20T10:23:14.948 INFO:tasks.ceph.osd.4.smithi088.stderr:2023-03-20T10:23:14.612+0000 7eff14993640 -1 osd.4 0 log_to_monitors true 2023-03-20T10:23:15.962 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:14.639+0000 7fe12834b640 -1 osd.6 0 log_to_monitors true 2023-03-20T10:23:15.963 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-20T10:23:15.963 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-20T10:23:15.963 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-20T10:23:14.529+0000 7f62b5437700 -1 osd.1 0 waiting for initial osdmap 2023-03-20T10:23:15.964 INFO:tasks.ceph.osd.1.smithi072.stderr:2023-03-20T10:23:14.544+0000 7f62ad812700 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:15.974 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-20T10:23:15.974+0000 7f2dc0b94700 -1 osd.3 0 waiting for initial osdmap 2023-03-20T10:23:20.930 INFO:tasks.ceph.osd.7.smithi088.stderr:2023-03-20T10:23:16.556+0000 7fdaf1aa0640 -1 osd.7 0 log_to_monitors true 2023-03-20T10:23:20.930 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-20T10:23:15.979+0000 7feb4d3ca700 -1 osd.2 0 waiting for initial osdmap 2023-03-20T10:23:20.930 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:17.364+0000 7f6e978df700 -1 osd.5 0 waiting for initial osdmap 2023-03-20T10:23:20.930 INFO:tasks.ceph.osd.3.smithi072.stderr:2023-03-20T10:23:15.985+0000 7f2db8f6f700 -1 osd.3 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:20.930 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:17.375+0000 7fe11bef1700 -1 osd.6 0 waiting for initial osdmap 2023-03-20T10:23:20.931 INFO:tasks.ceph.osd.2.smithi072.stderr:2023-03-20T10:23:15.994+0000 7feb457a5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:20.931 INFO:tasks.ceph.osd.5.smithi088.stderr:2023-03-20T10:23:17.376+0000 7f6e8fcba700 -1 osd.5 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:20.931 INFO:tasks.ceph.osd.6.smithi088.stderr:2023-03-20T10:23:17.390+0000 7fe1142cc700 -1 osd.6 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:20.931 INFO:tasks.ceph.osd.4.smithi088.stderr:2023-03-20T10:23:17.392+0000 7eff08539700 -1 osd.4 0 waiting for initial osdmap 2023-03-20T10:23:20.931 INFO:tasks.ceph.osd.4.smithi088.stderr:2023-03-20T10:23:17.407+0000 7eff00914700 -1 osd.4 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:20.932 INFO:tasks.ceph.osd.7.smithi088.stderr:2023-03-20T10:23:18.594+0000 7fdae5646700 -1 osd.7 0 waiting for initial osdmap 2023-03-20T10:23:20.932 INFO:tasks.ceph.osd.7.smithi088.stderr:2023-03-20T10:23:18.609+0000 7fdadda21700 -1 osd.7 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-20T10:23:21.634 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-20T10:23:23.099 INFO:teuthology.misc.health.smithi072.stdout: 2023-03-20T10:23:24.205 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":23,"fsid":"608e0299-7590-4702-bfbc-29e629ec4b60","created":"2023-03-20T10:22:32.881129+0000","modified":"2023-03-20T10:23:22.753800+0000","last_up_change":"2023-03-20T10:23:18.727809+0000","last_in_change":"2023-03-20T10:23:02.093998+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-20T10:23:16.570374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e2f177f-6f2a-4209-b131-5b35e996adc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":110509},{"type":"v1","addr":"172.21.15.72:6803","nonce":110509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":110509},{"type":"v1","addr":"172.21.15.72:6805","nonce":110509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":110509},{"type":"v1","addr":"172.21.15.72:6809","nonce":110509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":110509},{"type":"v1","addr":"172.21.15.72:6807","nonce":110509}]},"public_addr":"172.21.15.72:6803/110509","cluster_addr":"172.21.15.72:6805/110509","heartbeat_back_addr":"172.21.15.72:6809/110509","heartbeat_front_addr":"172.21.15.72:6807/110509","state":["exists","up"]},{"osd":1,"uuid":"93198d8a-c6c0-4743-a9a6-17e6cb4d57c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":111367},{"type":"v1","addr":"172.21.15.72:6811","nonce":111367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":111367},{"type":"v1","addr":"172.21.15.72:6813","nonce":111367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":111367},{"type":"v1","addr":"172.21.15.72:6817","nonce":111367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":111367},{"type":"v1","addr":"172.21.15.72:6815","nonce":111367}]},"public_addr":"172.21.15.72:6811/111367","cluster_addr":"172.21.15.72:6813/111367","heartbeat_back_addr":"172.21.15.72:6817/111367","heartbeat_front_addr":"172.21.15.72:6815/111367","state":["exists","up"]},{"osd":2,"uuid":"e38724c9-c316-48f0-8e50-3bb85e8c2eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":111442},{"type":"v1","addr":"172.21.15.72:6823","nonce":111442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":111442},{"type":"v1","addr":"172.21.15.72:6827","nonce":111442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":111442},{"type":"v1","addr":"172.21.15.72:6833","nonce":111442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":111442},{"type":"v1","addr":"172.21.15.72:6831","nonce":111442}]},"public_addr":"172.21.15.72:6823/111442","cluster_addr":"172.21.15.72:6827/111442","heartbeat_back_addr":"172.21.15.72:6833/111442","heartbeat_front_addr":"172.21.15.72:6831/111442","state":["exists","up"]},{"osd":3,"uuid":"fdbc2572-b14f-4014-a565-8563a0f026c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":111441},{"type":"v1","addr":"172.21.15.72:6819","nonce":111441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":111441},{"type":"v1","addr":"172.21.15.72:6822","nonce":111441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":111441},{"type":"v1","addr":"172.21.15.72:6830","nonce":111441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":111441},{"type":"v1","addr":"172.21.15.72:6826","nonce":111441}]},"public_addr":"172.21.15.72:6819/111441","cluster_addr":"172.21.15.72:6822/111441","heartbeat_back_addr":"172.21.15.72:6830/111441","heartbeat_front_addr":"172.21.15.72:6826/111441","state":["exists","up"]},{"osd":4,"uuid":"3b007991-2c19-45a3-9cc8-12e85291ba9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":110426},{"type":"v1","addr":"172.21.15.88:6802","nonce":110426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":110426},{"type":"v1","addr":"172.21.15.88:6808","nonce":110426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":110426},{"type":"v1","addr":"172.21.15.88:6821","nonce":110426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6811","nonce":110426},{"type":"v1","addr":"172.21.15.88:6815","nonce":110426}]},"public_addr":"172.21.15.88:6802/110426","cluster_addr":"172.21.15.88:6808/110426","heartbeat_back_addr":"172.21.15.88:6821/110426","heartbeat_front_addr":"172.21.15.88:6815/110426","state":["exists","up"]},{"osd":5,"uuid":"4feca5d3-6c51-47c2-a153-19d8d20e7047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6801","nonce":110425},{"type":"v1","addr":"172.21.15.88:6804","nonce":110425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6807","nonce":110425},{"type":"v1","addr":"172.21.15.88:6810","nonce":110425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":110425},{"type":"v1","addr":"172.21.15.88:6823","nonce":110425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":110425},{"type":"v1","addr":"172.21.15.88:6816","nonce":110425}]},"public_addr":"172.21.15.88:6804/110425","cluster_addr":"172.21.15.88:6810/110425","heartbeat_back_addr":"172.21.15.88:6823/110425","heartbeat_front_addr":"172.21.15.88:6816/110425","state":["exists","up"]},{"osd":6,"uuid":"5533656a-12bf-43ae-bfac-896b15840290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6803","nonce":110427},{"type":"v1","addr":"172.21.15.88:6806","nonce":110427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":110427},{"type":"v1","addr":"172.21.15.88:6812","nonce":110427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":110427},{"type":"v1","addr":"172.21.15.88:6822","nonce":110427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":110427},{"type":"v1","addr":"172.21.15.88:6817","nonce":110427}]},"public_addr":"172.21.15.88:6806/110427","cluster_addr":"172.21.15.88:6812/110427","heartbeat_back_addr":"172.21.15.88:6822/110427","heartbeat_front_addr":"172.21.15.88:6817/110427","state":["exists","up"]},{"osd":7,"uuid":"63b713e4-8511-4221-a5c1-1783e2b2b41c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":110549},{"type":"v1","addr":"172.21.15.88:6825","nonce":110549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":110549},{"type":"v1","addr":"172.21.15.88:6827","nonce":110549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":110549},{"type":"v1","addr":"172.21.15.88:6831","nonce":110549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":110549},{"type":"v1","addr":"172.21.15.88:6829","nonce":110549}]},"public_addr":"172.21.15.88:6825/110549","cluster_addr":"172.21.15.88:6827/110549","heartbeat_back_addr":"172.21.15.88:6831/110549","heartbeat_front_addr":"172.21.15.88:6829/110549","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:08.835491+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:13.036291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.080164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.001460+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.613693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.660464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.376082+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.576025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-20T10:23:24.825 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-20T10:23:24.825 INFO:tasks.ceph:Creating RBD pool 2023-03-20T10:23:24.826 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-20T10:23:25.779 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'rbd' created 2023-03-20T10:23:26.671 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-20T10:23:27.788 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-20T10:23:28.728 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-20T10:23:28.728 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-03-20T10:23:29.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-03-20T10:23:29.764 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-03-20T10:23:29.764 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-03-20T10:23:30.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:23:31.411 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":110192},{"type":"v1","addr":"172.21.15.72:6801","nonce":110192}]},"active_addr":"172.21.15.72:6801/110192","active_change":"2023-03-20T10:22:38.515427+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1142516553}]},{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1371198910}]},{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":118636758}]},{"addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2211156303}]}]} 2023-03-20T10:23:33.920 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-03-20T10:23:35.790 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-03-20T10:23:35.791 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-20T10:23:36.121 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:23:37.728 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"fsid":"608e0299-7590-4702-bfbc-29e629ec4b60","created":"2023-03-20T10:22:32.881129+0000","modified":"2023-03-20T10:23:27.778377+0000","last_up_change":"2023-03-20T10:23:18.727809+0000","last_in_change":"2023-03-20T10:23:02.093998+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-20T10:23:16.570374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-20T10:23:25.167545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e2f177f-6f2a-4209-b131-5b35e996adc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":110509},{"type":"v1","addr":"172.21.15.72:6803","nonce":110509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":110509},{"type":"v1","addr":"172.21.15.72:6805","nonce":110509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":110509},{"type":"v1","addr":"172.21.15.72:6809","nonce":110509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":110509},{"type":"v1","addr":"172.21.15.72:6807","nonce":110509}]},"public_addr":"172.21.15.72:6803/110509","cluster_addr":"172.21.15.72:6805/110509","heartbeat_back_addr":"172.21.15.72:6809/110509","heartbeat_front_addr":"172.21.15.72:6807/110509","state":["exists","up"]},{"osd":1,"uuid":"93198d8a-c6c0-4743-a9a6-17e6cb4d57c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":111367},{"type":"v1","addr":"172.21.15.72:6811","nonce":111367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":111367},{"type":"v1","addr":"172.21.15.72:6813","nonce":111367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":111367},{"type":"v1","addr":"172.21.15.72:6817","nonce":111367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":111367},{"type":"v1","addr":"172.21.15.72:6815","nonce":111367}]},"public_addr":"172.21.15.72:6811/111367","cluster_addr":"172.21.15.72:6813/111367","heartbeat_back_addr":"172.21.15.72:6817/111367","heartbeat_front_addr":"172.21.15.72:6815/111367","state":["exists","up"]},{"osd":2,"uuid":"e38724c9-c316-48f0-8e50-3bb85e8c2eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":111442},{"type":"v1","addr":"172.21.15.72:6823","nonce":111442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":111442},{"type":"v1","addr":"172.21.15.72:6827","nonce":111442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":111442},{"type":"v1","addr":"172.21.15.72:6833","nonce":111442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":111442},{"type":"v1","addr":"172.21.15.72:6831","nonce":111442}]},"public_addr":"172.21.15.72:6823/111442","cluster_addr":"172.21.15.72:6827/111442","heartbeat_back_addr":"172.21.15.72:6833/111442","heartbeat_front_addr":"172.21.15.72:6831/111442","state":["exists","up"]},{"osd":3,"uuid":"fdbc2572-b14f-4014-a565-8563a0f026c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":111441},{"type":"v1","addr":"172.21.15.72:6819","nonce":111441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":111441},{"type":"v1","addr":"172.21.15.72:6822","nonce":111441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":111441},{"type":"v1","addr":"172.21.15.72:6830","nonce":111441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":111441},{"type":"v1","addr":"172.21.15.72:6826","nonce":111441}]},"public_addr":"172.21.15.72:6819/111441","cluster_addr":"172.21.15.72:6822/111441","heartbeat_back_addr":"172.21.15.72:6830/111441","heartbeat_front_addr":"172.21.15.72:6826/111441","state":["exists","up"]},{"osd":4,"uuid":"3b007991-2c19-45a3-9cc8-12e85291ba9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":110426},{"type":"v1","addr":"172.21.15.88:6802","nonce":110426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":110426},{"type":"v1","addr":"172.21.15.88:6808","nonce":110426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":110426},{"type":"v1","addr":"172.21.15.88:6821","nonce":110426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6811","nonce":110426},{"type":"v1","addr":"172.21.15.88:6815","nonce":110426}]},"public_addr":"172.21.15.88:6802/110426","cluster_addr":"172.21.15.88:6808/110426","heartbeat_back_addr":"172.21.15.88:6821/110426","heartbeat_front_addr":"172.21.15.88:6815/110426","state":["exists","up"]},{"osd":5,"uuid":"4feca5d3-6c51-47c2-a153-19d8d20e7047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6801","nonce":110425},{"type":"v1","addr":"172.21.15.88:6804","nonce":110425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6807","nonce":110425},{"type":"v1","addr":"172.21.15.88:6810","nonce":110425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":110425},{"type":"v1","addr":"172.21.15.88:6823","nonce":110425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":110425},{"type":"v1","addr":"172.21.15.88:6816","nonce":110425}]},"public_addr":"172.21.15.88:6804/110425","cluster_addr":"172.21.15.88:6810/110425","heartbeat_back_addr":"172.21.15.88:6823/110425","heartbeat_front_addr":"172.21.15.88:6816/110425","state":["exists","up"]},{"osd":6,"uuid":"5533656a-12bf-43ae-bfac-896b15840290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6803","nonce":110427},{"type":"v1","addr":"172.21.15.88:6806","nonce":110427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":110427},{"type":"v1","addr":"172.21.15.88:6812","nonce":110427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":110427},{"type":"v1","addr":"172.21.15.88:6822","nonce":110427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":110427},{"type":"v1","addr":"172.21.15.88:6817","nonce":110427}]},"public_addr":"172.21.15.88:6806/110427","cluster_addr":"172.21.15.88:6812/110427","heartbeat_back_addr":"172.21.15.88:6822/110427","heartbeat_front_addr":"172.21.15.88:6817/110427","state":["exists","up"]},{"osd":7,"uuid":"63b713e4-8511-4221-a5c1-1783e2b2b41c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":110549},{"type":"v1","addr":"172.21.15.88:6825","nonce":110549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":110549},{"type":"v1","addr":"172.21.15.88:6827","nonce":110549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":110549},{"type":"v1","addr":"172.21.15.88:6831","nonce":110549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":110549},{"type":"v1","addr":"172.21.15.88:6829","nonce":110549}]},"public_addr":"172.21.15.88:6825/110549","cluster_addr":"172.21.15.88:6827/110549","heartbeat_back_addr":"172.21.15.88:6831/110549","heartbeat_front_addr":"172.21.15.88:6829/110549","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:08.835491+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:13.036291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.080164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.001460+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.613693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.660464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.376082+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.576025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-20T10:23:37.730 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-03-20T10:23:39.254 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-20T10:23:39.581 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:23:40.875 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":27,"fsid":"608e0299-7590-4702-bfbc-29e629ec4b60","created":"2023-03-20T10:22:32.881129+0000","modified":"2023-03-20T10:23:27.778377+0000","last_up_change":"2023-03-20T10:23:18.727809+0000","last_in_change":"2023-03-20T10:23:02.093998+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-20T10:23:16.570374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-20T10:23:25.167545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e2f177f-6f2a-4209-b131-5b35e996adc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":110509},{"type":"v1","addr":"172.21.15.72:6803","nonce":110509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":110509},{"type":"v1","addr":"172.21.15.72:6805","nonce":110509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":110509},{"type":"v1","addr":"172.21.15.72:6809","nonce":110509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":110509},{"type":"v1","addr":"172.21.15.72:6807","nonce":110509}]},"public_addr":"172.21.15.72:6803/110509","cluster_addr":"172.21.15.72:6805/110509","heartbeat_back_addr":"172.21.15.72:6809/110509","heartbeat_front_addr":"172.21.15.72:6807/110509","state":["exists","up"]},{"osd":1,"uuid":"93198d8a-c6c0-4743-a9a6-17e6cb4d57c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":111367},{"type":"v1","addr":"172.21.15.72:6811","nonce":111367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":111367},{"type":"v1","addr":"172.21.15.72:6813","nonce":111367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":111367},{"type":"v1","addr":"172.21.15.72:6817","nonce":111367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":111367},{"type":"v1","addr":"172.21.15.72:6815","nonce":111367}]},"public_addr":"172.21.15.72:6811/111367","cluster_addr":"172.21.15.72:6813/111367","heartbeat_back_addr":"172.21.15.72:6817/111367","heartbeat_front_addr":"172.21.15.72:6815/111367","state":["exists","up"]},{"osd":2,"uuid":"e38724c9-c316-48f0-8e50-3bb85e8c2eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6821","nonce":111442},{"type":"v1","addr":"172.21.15.72:6823","nonce":111442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6825","nonce":111442},{"type":"v1","addr":"172.21.15.72:6827","nonce":111442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":111442},{"type":"v1","addr":"172.21.15.72:6833","nonce":111442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6829","nonce":111442},{"type":"v1","addr":"172.21.15.72:6831","nonce":111442}]},"public_addr":"172.21.15.72:6823/111442","cluster_addr":"172.21.15.72:6827/111442","heartbeat_back_addr":"172.21.15.72:6833/111442","heartbeat_front_addr":"172.21.15.72:6831/111442","state":["exists","up"]},{"osd":3,"uuid":"fdbc2572-b14f-4014-a565-8563a0f026c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":111441},{"type":"v1","addr":"172.21.15.72:6819","nonce":111441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":111441},{"type":"v1","addr":"172.21.15.72:6822","nonce":111441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":111441},{"type":"v1","addr":"172.21.15.72:6830","nonce":111441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":111441},{"type":"v1","addr":"172.21.15.72:6826","nonce":111441}]},"public_addr":"172.21.15.72:6819/111441","cluster_addr":"172.21.15.72:6822/111441","heartbeat_back_addr":"172.21.15.72:6830/111441","heartbeat_front_addr":"172.21.15.72:6826/111441","state":["exists","up"]},{"osd":4,"uuid":"3b007991-2c19-45a3-9cc8-12e85291ba9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":110426},{"type":"v1","addr":"172.21.15.88:6802","nonce":110426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6805","nonce":110426},{"type":"v1","addr":"172.21.15.88:6808","nonce":110426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":110426},{"type":"v1","addr":"172.21.15.88:6821","nonce":110426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6811","nonce":110426},{"type":"v1","addr":"172.21.15.88:6815","nonce":110426}]},"public_addr":"172.21.15.88:6802/110426","cluster_addr":"172.21.15.88:6808/110426","heartbeat_back_addr":"172.21.15.88:6821/110426","heartbeat_front_addr":"172.21.15.88:6815/110426","state":["exists","up"]},{"osd":5,"uuid":"4feca5d3-6c51-47c2-a153-19d8d20e7047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6801","nonce":110425},{"type":"v1","addr":"172.21.15.88:6804","nonce":110425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6807","nonce":110425},{"type":"v1","addr":"172.21.15.88:6810","nonce":110425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":110425},{"type":"v1","addr":"172.21.15.88:6823","nonce":110425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":110425},{"type":"v1","addr":"172.21.15.88:6816","nonce":110425}]},"public_addr":"172.21.15.88:6804/110425","cluster_addr":"172.21.15.88:6810/110425","heartbeat_back_addr":"172.21.15.88:6823/110425","heartbeat_front_addr":"172.21.15.88:6816/110425","state":["exists","up"]},{"osd":6,"uuid":"5533656a-12bf-43ae-bfac-896b15840290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6803","nonce":110427},{"type":"v1","addr":"172.21.15.88:6806","nonce":110427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":110427},{"type":"v1","addr":"172.21.15.88:6812","nonce":110427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":110427},{"type":"v1","addr":"172.21.15.88:6822","nonce":110427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":110427},{"type":"v1","addr":"172.21.15.88:6817","nonce":110427}]},"public_addr":"172.21.15.88:6806/110427","cluster_addr":"172.21.15.88:6812/110427","heartbeat_back_addr":"172.21.15.88:6822/110427","heartbeat_front_addr":"172.21.15.88:6817/110427","state":["exists","up"]},{"osd":7,"uuid":"63b713e4-8511-4221-a5c1-1783e2b2b41c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":110549},{"type":"v1","addr":"172.21.15.88:6825","nonce":110549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":110549},{"type":"v1","addr":"172.21.15.88:6827","nonce":110549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":110549},{"type":"v1","addr":"172.21.15.88:6831","nonce":110549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":110549},{"type":"v1","addr":"172.21.15.88:6829","nonce":110549}]},"public_addr":"172.21.15.88:6825/110549","cluster_addr":"172.21.15.88:6827/110549","heartbeat_back_addr":"172.21.15.88:6831/110549","heartbeat_front_addr":"172.21.15.88:6829/110549","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:08.835491+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:13.036291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.080164+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:14.001460+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.613693+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:15.660464+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.376082+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-20T10:23:17.576025+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-20T10:23:40.877 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-03-20T10:23:42.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-03-20T10:23:42.167 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-03-20T10:23:42.168 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-03-20T10:23:42.168 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-03-20T10:23:42.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-03-20T10:23:42.169 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-03-20T10:23:42.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-03-20T10:23:42.457 INFO:teuthology.orchestra.run.smithi072.stdout:81604378632 2023-03-20T10:23:43.876 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-03-20T10:23:43.878 INFO:teuthology.orchestra.run.smithi072.stdout:81604378633 2023-03-20T10:23:45.706 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-03-20T10:23:45.706 INFO:teuthology.orchestra.run.smithi072.stdout:81604378633 2023-03-20T10:23:45.707 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-03-20T10:23:45.708 INFO:teuthology.orchestra.run.smithi072.stdout:68719476745 2023-03-20T10:23:46.997 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-20T10:23:48.455 INFO:teuthology.orchestra.run.smithi072.stdout:77309411337 2023-03-20T10:23:48.456 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-20T10:23:48.457 INFO:teuthology.orchestra.run.smithi072.stdout:77309411336 2023-03-20T10:23:48.457 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-20T10:23:48.458 INFO:teuthology.orchestra.run.smithi072.stdout:85899345926 2023-03-20T10:23:48.458 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-03-20T10:23:48.459 INFO:teuthology.orchestra.run.smithi072.stdout:55834574858 2023-03-20T10:23:48.460 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-20T10:23:49.126 INFO:teuthology.orchestra.run.smithi072.stdout:81604378634 2023-03-20T10:23:53.322 INFO:teuthology.orchestra.run.smithi072.stdout:68719476746 2023-03-20T10:23:53.322 INFO:teuthology.orchestra.run.smithi072.stdout:81604378634 2023-03-20T10:23:53.323 INFO:teuthology.orchestra.run.smithi072.stdout:77309411339 2023-03-20T10:23:54.662 INFO:teuthology.orchestra.run.smithi072.stdout:81604378633 2023-03-20T10:23:54.664 INFO:teuthology.orchestra.run.smithi072.stdout:77309411338 2023-03-20T10:23:54.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378633 got 81604378634 for osd.5 2023-03-20T10:23:55.656 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:55.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378633 got 81604378634 for osd.4 2023-03-20T10:23:55.657 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:56.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411337 got 77309411339 for osd.3 2023-03-20T10:23:57.758 INFO:teuthology.orchestra.run.smithi072.stdout:55834574859 2023-03-20T10:23:59.056 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:59.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476746 for osd.1 2023-03-20T10:23:59.057 INFO:teuthology.orchestra.run.smithi072.stdout:85899345927 2023-03-20T10:23:59.057 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:59.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 81604378632 got 81604378633 for osd.6 2023-03-20T10:23:59.058 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:59.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411336 got 77309411338 for osd.2 2023-03-20T10:23:59.060 DEBUG:teuthology.parallel:result is None 2023-03-20T10:23:59.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.0 2023-03-20T10:24:02.116 DEBUG:teuthology.parallel:result is None 2023-03-20T10:24:02.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 85899345926 got 85899345927 for osd.7 2023-03-20T10:24:04.797 DEBUG:teuthology.parallel:result is None 2023-03-20T10:24:04.797 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-03-20T10:24:04.797 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-20T10:24:05.122 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:24:09.717 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-03-20T10:24:09.717 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-03-20T10:24:04.534271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":64936,"kb_used_data":3240,"kb_used_omap":0,"kb_used_meta":61632,"kb_avail":754909784,"statfs":{"total":773094113280,"available":773027618816,"internally_reserved":0,"allocated":3317760,"data_stored":1929008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63111168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.683930+0000","last_change":"2023-03-20T10:23:26.782588+0000","last_active":"2023-03-20T10:23:28.683930+0000","last_peered":"2023-03-20T10:23:28.683930+0000","last_clean":"2023-03-20T10:23:28.683930+0000","last_became_active":"2023-03-20T10:23:26.782260+0000","last_became_peered":"2023-03-20T10:23:26.782260+0000","last_unstale":"2023-03-20T10:23:28.683930+0000","last_undegraded":"2023-03-20T10:23:28.683930+0000","last_fullsized":"2023-03-20T10:23:28.683930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T19:58:02.972540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.757137+0000","last_change":"2023-03-20T10:23:26.786390+0000","last_active":"2023-03-20T10:23:28.757137+0000","last_peered":"2023-03-20T10:23:28.757137+0000","last_clean":"2023-03-20T10:23:28.757137+0000","last_became_active":"2023-03-20T10:23:26.786080+0000","last_became_peered":"2023-03-20T10:23:26.786080+0000","last_unstale":"2023-03-20T10:23:28.757137+0000","last_undegraded":"2023-03-20T10:23:28.757137+0000","last_fullsized":"2023-03-20T10:23:28.757137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T12:44:20.469841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.782368+0000","last_change":"2023-03-20T10:23:26.829648+0000","last_active":"2023-03-20T10:23:27.782368+0000","last_peered":"2023-03-20T10:23:27.782368+0000","last_clean":"2023-03-20T10:23:27.782368+0000","last_became_active":"2023-03-20T10:23:26.829326+0000","last_became_peered":"2023-03-20T10:23:26.829326+0000","last_unstale":"2023-03-20T10:23:27.782368+0000","last_undegraded":"2023-03-20T10:23:27.782368+0000","last_fullsized":"2023-03-20T10:23:27.782368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T11:35:34.820441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.782378+0000","last_change":"2023-03-20T10:23:26.786926+0000","last_active":"2023-03-20T10:23:27.782378+0000","last_peered":"2023-03-20T10:23:27.782378+0000","last_clean":"2023-03-20T10:23:27.782378+0000","last_became_active":"2023-03-20T10:23:26.786478+0000","last_became_peered":"2023-03-20T10:23:26.786478+0000","last_unstale":"2023-03-20T10:23:27.782378+0000","last_undegraded":"2023-03-20T10:23:27.782378+0000","last_fullsized":"2023-03-20T10:23:27.782378+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T17:57:29.649630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":183,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:26.780579+0000","last_change":"2023-03-20T10:23:23.764089+0000","last_active":"2023-03-20T10:23:26.780579+0000","last_peered":"2023-03-20T10:23:26.780579+0000","last_clean":"2023-03-20T10:23:26.780579+0000","last_became_active":"2023-03-20T10:23:23.763797+0000","last_became_peered":"2023-03-20T10:23:23.763797+0000","last_unstale":"2023-03-20T10:23:26.780579+0000","last_undegraded":"2023-03-20T10:23:26.780579+0000","last_fullsized":"2023-03-20T10:23:26.780579+0000","mapping_epoch":23,"log_start":"19'107","ondisk_log_start":"19'107","created":18,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:16.673125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:16.673125+0000","last_clean_scrub_stamp":"2023-03-20T10:23:16.673125+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":107,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T21:32:02.377025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.676786+0000","last_change":"2023-03-20T10:23:26.788031+0000","last_active":"2023-03-20T10:23:28.676786+0000","last_peered":"2023-03-20T10:23:28.676786+0000","last_clean":"2023-03-20T10:23:28.676786+0000","last_became_active":"2023-03-20T10:23:26.787780+0000","last_became_peered":"2023-03-20T10:23:26.787780+0000","last_unstale":"2023-03-20T10:23:28.676786+0000","last_undegraded":"2023-03-20T10:23:28.676786+0000","last_fullsized":"2023-03-20T10:23:28.676786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T21:15:31.877360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.667219+0000","last_change":"2023-03-20T10:23:26.847537+0000","last_active":"2023-03-20T10:23:27.667219+0000","last_peered":"2023-03-20T10:23:27.667219+0000","last_clean":"2023-03-20T10:23:27.667219+0000","last_became_active":"2023-03-20T10:23:26.847270+0000","last_became_peered":"2023-03-20T10:23:26.847270+0000","last_unstale":"2023-03-20T10:23:27.667219+0000","last_undegraded":"2023-03-20T10:23:27.667219+0000","last_fullsized":"2023-03-20T10:23:27.667219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T10:42:02.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.676744+0000","last_change":"2023-03-20T10:23:26.787243+0000","last_active":"2023-03-20T10:23:28.676744+0000","last_peered":"2023-03-20T10:23:28.676744+0000","last_clean":"2023-03-20T10:23:28.676744+0000","last_became_active":"2023-03-20T10:23:26.786967+0000","last_became_peered":"2023-03-20T10:23:26.786967+0000","last_unstale":"2023-03-20T10:23:28.676744+0000","last_undegraded":"2023-03-20T10:23:28.676744+0000","last_fullsized":"2023-03-20T10:23:28.676744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T13:28:04.486307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.757102+0000","last_change":"2023-03-20T10:23:26.829494+0000","last_active":"2023-03-20T10:23:28.757102+0000","last_peered":"2023-03-20T10:23:28.757102+0000","last_clean":"2023-03-20T10:23:28.757102+0000","last_became_active":"2023-03-20T10:23:26.829148+0000","last_became_peered":"2023-03-20T10:23:26.829148+0000","last_unstale":"2023-03-20T10:23:28.757102+0000","last_undegraded":"2023-03-20T10:23:28.757102+0000","last_fullsized":"2023-03-20T10:23:28.757102+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T15:33:45.730641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":20,"seq":85899345931,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":860160,"data_stored":683902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":6,"up_from":19,"seq":81604378637,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476749,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7948,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363892,"statfs":{"total":96636764160,"available":96628625408,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9040,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94362800,"statfs":{"total":96636764160,"available":96627507200,"internally_reserved":0,"allocated":860160,"data_stored":683902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411341,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":3,"up_from":18,"seq":77309411342,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]}]},{"osd":4,"up_from":19,"seq":81604378638,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]}]},{"osd":5,"up_from":19,"seq":81604378638,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-20T10:24:10.779 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-20T10:24:11.106 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:24:12.766 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2023-03-20T10:24:13.775 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-03-20T10:24:10.535472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":64936,"kb_used_data":3240,"kb_used_omap":0,"kb_used_meta":61632,"kb_avail":754909784,"statfs":{"total":773094113280,"available":773027618816,"internally_reserved":0,"allocated":3317760,"data_stored":1929008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":63111168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002583"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.683930+0000","last_change":"2023-03-20T10:23:26.782588+0000","last_active":"2023-03-20T10:23:28.683930+0000","last_peered":"2023-03-20T10:23:28.683930+0000","last_clean":"2023-03-20T10:23:28.683930+0000","last_became_active":"2023-03-20T10:23:26.782260+0000","last_became_peered":"2023-03-20T10:23:26.782260+0000","last_unstale":"2023-03-20T10:23:28.683930+0000","last_undegraded":"2023-03-20T10:23:28.683930+0000","last_fullsized":"2023-03-20T10:23:28.683930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T19:58:02.972540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.757137+0000","last_change":"2023-03-20T10:23:26.786390+0000","last_active":"2023-03-20T10:23:28.757137+0000","last_peered":"2023-03-20T10:23:28.757137+0000","last_clean":"2023-03-20T10:23:28.757137+0000","last_became_active":"2023-03-20T10:23:26.786080+0000","last_became_peered":"2023-03-20T10:23:26.786080+0000","last_unstale":"2023-03-20T10:23:28.757137+0000","last_undegraded":"2023-03-20T10:23:28.757137+0000","last_fullsized":"2023-03-20T10:23:28.757137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T12:44:20.469841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.782368+0000","last_change":"2023-03-20T10:23:26.829648+0000","last_active":"2023-03-20T10:23:27.782368+0000","last_peered":"2023-03-20T10:23:27.782368+0000","last_clean":"2023-03-20T10:23:27.782368+0000","last_became_active":"2023-03-20T10:23:26.829326+0000","last_became_peered":"2023-03-20T10:23:26.829326+0000","last_unstale":"2023-03-20T10:23:27.782368+0000","last_undegraded":"2023-03-20T10:23:27.782368+0000","last_fullsized":"2023-03-20T10:23:27.782368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T11:35:34.820441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.782378+0000","last_change":"2023-03-20T10:23:26.786926+0000","last_active":"2023-03-20T10:23:27.782378+0000","last_peered":"2023-03-20T10:23:27.782378+0000","last_clean":"2023-03-20T10:23:27.782378+0000","last_became_active":"2023-03-20T10:23:26.786478+0000","last_became_peered":"2023-03-20T10:23:26.786478+0000","last_unstale":"2023-03-20T10:23:27.782378+0000","last_undegraded":"2023-03-20T10:23:27.782378+0000","last_fullsized":"2023-03-20T10:23:27.782378+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T17:57:29.649630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":183,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:26.780579+0000","last_change":"2023-03-20T10:23:23.764089+0000","last_active":"2023-03-20T10:23:26.780579+0000","last_peered":"2023-03-20T10:23:26.780579+0000","last_clean":"2023-03-20T10:23:26.780579+0000","last_became_active":"2023-03-20T10:23:23.763797+0000","last_became_peered":"2023-03-20T10:23:23.763797+0000","last_unstale":"2023-03-20T10:23:26.780579+0000","last_undegraded":"2023-03-20T10:23:26.780579+0000","last_fullsized":"2023-03-20T10:23:26.780579+0000","mapping_epoch":23,"log_start":"19'107","ondisk_log_start":"19'107","created":18,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:16.673125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:16.673125+0000","last_clean_scrub_stamp":"2023-03-20T10:23:16.673125+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":107,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T21:32:02.377025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.676786+0000","last_change":"2023-03-20T10:23:26.788031+0000","last_active":"2023-03-20T10:23:28.676786+0000","last_peered":"2023-03-20T10:23:28.676786+0000","last_clean":"2023-03-20T10:23:28.676786+0000","last_became_active":"2023-03-20T10:23:26.787780+0000","last_became_peered":"2023-03-20T10:23:26.787780+0000","last_unstale":"2023-03-20T10:23:28.676786+0000","last_undegraded":"2023-03-20T10:23:28.676786+0000","last_fullsized":"2023-03-20T10:23:28.676786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T21:15:31.877360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:27.667219+0000","last_change":"2023-03-20T10:23:26.847537+0000","last_active":"2023-03-20T10:23:27.667219+0000","last_peered":"2023-03-20T10:23:27.667219+0000","last_clean":"2023-03-20T10:23:27.667219+0000","last_became_active":"2023-03-20T10:23:26.847270+0000","last_became_peered":"2023-03-20T10:23:26.847270+0000","last_unstale":"2023-03-20T10:23:27.667219+0000","last_undegraded":"2023-03-20T10:23:27.667219+0000","last_fullsized":"2023-03-20T10:23:27.667219+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T10:42:02.550134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":26,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.676744+0000","last_change":"2023-03-20T10:23:26.787243+0000","last_active":"2023-03-20T10:23:28.676744+0000","last_peered":"2023-03-20T10:23:28.676744+0000","last_clean":"2023-03-20T10:23:28.676744+0000","last_became_active":"2023-03-20T10:23:26.786967+0000","last_became_peered":"2023-03-20T10:23:26.786967+0000","last_unstale":"2023-03-20T10:23:28.676744+0000","last_undegraded":"2023-03-20T10:23:28.676744+0000","last_fullsized":"2023-03-20T10:23:28.676744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T13:28:04.486307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-20T10:23:28.757102+0000","last_change":"2023-03-20T10:23:26.829494+0000","last_active":"2023-03-20T10:23:28.757102+0000","last_peered":"2023-03-20T10:23:28.757102+0000","last_clean":"2023-03-20T10:23:28.757102+0000","last_became_active":"2023-03-20T10:23:26.829148+0000","last_became_peered":"2023-03-20T10:23:26.829148+0000","last_unstale":"2023-03-20T10:23:28.757102+0000","last_undegraded":"2023-03-20T10:23:28.757102+0000","last_fullsized":"2023-03-20T10:23:28.757102+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-20T10:23:25.769367+0000","last_clean_scrub_stamp":"2023-03-20T10:23:25.769367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-21T15:33:45.730641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":20,"seq":85899345932,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":860160,"data_stored":683902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":6,"up_from":19,"seq":81604378638,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":1,"up_from":16,"seq":68719476750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7948,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7680,"kb_avail":94363892,"statfs":{"total":96636764160,"available":96628625408,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7864320},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9040,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94362800,"statfs":{"total":96636764160,"available":96627507200,"internally_reserved":0,"allocated":860160,"data_stored":683902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":3,"up_from":18,"seq":77309411343,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]}]},{"osd":4,"up_from":19,"seq":81604378639,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":5,"up_from":19,"seq":81604378639,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7884,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7616,"kb_avail":94363956,"statfs":{"total":96636764160,"available":96628690944,"internally_reserved":0,"allocated":266240,"data_stored":93534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7798784},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-20T10:24:15.465 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-03-20T10:24:16.020 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-03-20T10:24:16.020 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-03-20T10:24:16.020 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-03-20T10:24:17.841 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:24:19.199 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-03-20T10:24:19.200 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-03-20T10:24:19.200 INFO:teuthology.run_tasks:Running task openssl_keys... 2023-03-20T10:24:19.224 INFO:teuthology.run_tasks:Running task rgw... 2023-03-20T10:24:24.003 DEBUG:tasks.rgw:config is {'client.0': None} 2023-03-20T10:24:25.629 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2023-03-20T10:24:25.629 INFO:tasks.rgw:Creating data pools 2023-03-20T10:24:25.629 DEBUG:tasks.rgw:Obtaining remote for client client.0 2023-03-20T10:24:25.629 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2023-03-20T10:24:26.943 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'default.rgw.buckets.data' created 2023-03-20T10:24:28.931 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2023-03-20T10:24:29.957 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2023-03-20T10:24:31.460 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2023-03-20T10:24:32.658 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'default.rgw.buckets.index' created 2023-03-20T10:24:33.240 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2023-03-20T10:24:33.670 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2023-03-20T10:24:36.302 DEBUG:tasks.rgw:Pools created 2023-03-20T10:24:36.713 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2023-03-20T10:24:36.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2023-03-20T10:24:36.713 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2023-03-20T10:24:36.848 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:24:38.093 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:24:38.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:36.861+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:38.701 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.700+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.700+0000 7fd86a7b3780 20 realm 2023-03-20T10:24:40.402 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.700+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.701+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.701+0000 7fd86a7b3780 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-03-20T10:24:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.701+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.701+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.403 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.701+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.702+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.702+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.702+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.702+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.703+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.703+0000 7fd86a7b3780 10 Creating default zonegroup 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.703+0000 7fd86a7b3780 10 couldn't find old data placement pools config, setting up new ones for the zone 2023-03-20T10:24:40.405 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.716+0000 7fd86a7b3780 10 WARNING: driver->list_zones() returned r=-2 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.716+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.716+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.719+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.719+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.719+0000 7fd86a7b3780 10 could not read realm id: (2) No such file or directory 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.719+0000 7fd86a7b3780 10 WARNING: failed to set zone as default, r=-22 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.719+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.720+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.724+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.725+0000 7fd86a7b3780 20 rados_obj.operate() r=0 bl.length=874 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.725+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.725+0000 7fd86a7b3780 20 rados_obj.operate() r=0 bl.length=398 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.725+0000 7fd86a7b3780 10 Using default name default 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.725+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados_obj.operate() r=0 bl.length=46 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados_obj.operate() r=0 bl.length=874 2023-03-20T10:24:40.406 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 zone default found 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 4 Realm: () 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 4 ZoneGroup: default (2d432b72-35d2-4754-a1e2-8b1c802e34d5) 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 4 Zone: default (a09596be-35b9-426c-8b99-1da61aeeb17e) 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 10 cannot find current period zonegroup using local zonegroup configuration 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 zonegroup default 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:40.407 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:38.726+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:40.710 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:40.709+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:42.442 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:40.709+0000 7fd86a7b3780 20 rados->read ofs=0 len=0 2023-03-20T10:24:42.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:40.710+0000 7fd86a7b3780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:24:42.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:40.710+0000 7fd86a7b3780 20 started sync module instance, tier type = 2023-03-20T10:24:42.443 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:40.710+0000 7fd86a7b3780 20 started zone id=a09596be-35b9-426c-8b99-1da61aeeb17e (name=default) with tier type = 2023-03-20T10:24:42.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=0 2023-03-20T10:24:43.469 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=1 2023-03-20T10:24:43.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=2 2023-03-20T10:24:43.470 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=3 2023-03-20T10:24:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=4 2023-03-20T10:24:44.692 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=5 2023-03-20T10:24:44.693 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.738+0000 7fd86a7b3780 20 add_watcher() i=6 2023-03-20T10:24:44.704 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.741+0000 7fd86a7b3780 20 add_watcher() i=7 2023-03-20T10:24:47.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.741+0000 7fd86a7b3780 2 all 8 watchers are set, enabling cache 2023-03-20T10:24:47.777 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.741+0000 7fd86a7b3780 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2023-03-20T10:24:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.741+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2023-03-20T10:24:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.741+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2023-03-20T10:24:47.778 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2023-03-20T10:24:48.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2023-03-20T10:24:48.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2023-03-20T10:24:48.591 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.742+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.592 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2023-03-20T10:24:48.593 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.743+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.744+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2023-03-20T10:24:48.594 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.745+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:48.595 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2023-03-20T10:24:50.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2023-03-20T10:24:50.737 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:50.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2023-03-20T10:24:50.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2023-03-20T10:24:50.738 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.746+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2023-03-20T10:24:51.312 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.747+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2023-03-20T10:24:51.313 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2023-03-20T10:24:51.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:51.314 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2023-03-20T10:24:55.159 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2023-03-20T10:24:55.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2023-03-20T10:24:55.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2023-03-20T10:24:55.160 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.748+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2023-03-20T10:24:55.812 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2023-03-20T10:24:55.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2023-03-20T10:24:55.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2023-03-20T10:24:55.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:55.813 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2023-03-20T10:24:57.024 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.749+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2023-03-20T10:24:57.025 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2023-03-20T10:24:57.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:57.026 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.750+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.239 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.751+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2023-03-20T10:24:58.827 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.752+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.753+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2023-03-20T10:24:58.828 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2023-03-20T10:24:58.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.829 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.754+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2023-03-20T10:24:58.830 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.755+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.756+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2023-03-20T10:24:58.831 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.757+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2023-03-20T10:24:58.832 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2023-03-20T10:24:58.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:24:58.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2023-03-20T10:24:58.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2023-03-20T10:24:58.833 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:01.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2023-03-20T10:25:01.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2023-03-20T10:25:01.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.758+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:01.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2023-03-20T10:25:02.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2023-03-20T10:25:02.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:02.300 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2023-03-20T10:25:02.301 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:02.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2023-03-20T10:25:02.302 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.759+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2023-03-20T10:25:04.092 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.760+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.761+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.093 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.762+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.094 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.763+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.095 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.764+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2023-03-20T10:25:04.096 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2023-03-20T10:25:04.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:04.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2023-03-20T10:25:04.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2023-03-20T10:25:04.097 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:05.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2023-03-20T10:25:05.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2023-03-20T10:25:05.180 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.765+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2023-03-20T10:25:05.181 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:05.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2023-03-20T10:25:05.182 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2023-03-20T10:25:06.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:06.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2023-03-20T10:25:06.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2023-03-20T10:25:06.814 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.766+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:07.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2023-03-20T10:25:07.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2023-03-20T10:25:07.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:07.633 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.767+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:08.252 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2023-03-20T10:25:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2023-03-20T10:25:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:08.253 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2023-03-20T10:25:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2023-03-20T10:25:09.068 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.768+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:09.069 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2023-03-20T10:25:09.865 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2023-03-20T10:25:10.650 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.769+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:10.969 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.770+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2023-03-20T10:25:11.505 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.771+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2023-03-20T10:25:11.506 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.772+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2023-03-20T10:25:11.508 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2023-03-20T10:25:12.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.320 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.773+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.321 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.651 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.774+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2023-03-20T10:25:12.652 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:13.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2023-03-20T10:25:13.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2023-03-20T10:25:13.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:13.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2023-03-20T10:25:13.142 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.775+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.776+0000 7fd86a7b3780 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2023-03-20T10:25:13.143 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.776+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1210 entering 2023-03-20T10:25:14.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.777+0000 7fd86a7b3780 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1168 entering 2023-03-20T10:25:14.292 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.782+0000 7fd84f72f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-03-20T10:25:14.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.782+0000 7fd86a7b3780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:25:14.787 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:42.782+0000 7fd86a7b3780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:25:14.789 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.707+0000 7fd86a7b3780 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.707+0000 7fd86a7b3780 5 note: GC not initialized 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.708+0000 7fd84af26700 20 reqs_thread_entry: start 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.708+0000 7fd86a7b3780 20 init_complete bucket index max shards: 11 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.708+0000 7fd86a7b3780 10 rgw notify: Started notification manager with: 1 workers 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.708+0000 7fd848f22700 20 rgw notify: INFO: next queues processing will happen at: Mon Mar 20 10:25:15 2023 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.708+0000 7fd848700700 20 reqs_thread_entry: start 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.731+0000 7fd84f72f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.734+0000 7fd86a7b3780 20 remove_watcher() i=0 2023-03-20T10:25:16.351 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.734+0000 7fd86a7b3780 2 removed watcher, disabling cache 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.735+0000 7fd86a7b3780 20 remove_watcher() i=1 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.737+0000 7fd86a7b3780 20 remove_watcher() i=2 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.741+0000 7fd86a7b3780 20 remove_watcher() i=3 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.742+0000 7fd86a7b3780 20 remove_watcher() i=4 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.743+0000 7fd86a7b3780 20 remove_watcher() i=5 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.744+0000 7fd86a7b3780 20 remove_watcher() i=6 2023-03-20T10:25:16.352 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:24:45.745+0000 7fd86a7b3780 20 remove_watcher() i=7 2023-03-20T10:25:16.353 INFO:teuthology.orchestra.run.smithi072.stdout:[] 2023-03-20T10:25:16.353 DEBUG:tasks.util.rgw: json result: [] 2023-03-20T10:25:18.651 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2023-03-20T10:25:18.651 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2023-03-20T10:25:18.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2023-03-20T10:25:18.651 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2023-03-20T10:25:18.710 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:25:20.170 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:25:20.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:18.719+0000 7f7157f9f780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:25:20.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:18.719+0000 7f7157f9f780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:25:20.171 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:18.724+0000 7f7139714700 20 reqs_thread_entry: start 2023-03-20T10:25:20.172 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-03-20T10:25:20.173 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-20T10:25:20.173 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "default-placement", 2023-03-20T10:25:20.606 INFO:teuthology.orchestra.run.smithi072.stdout: "val": { 2023-03-20T10:25:20.606 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default-placement", 2023-03-20T10:25:20.606 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2023-03-20T10:25:20.606 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_classes": [ 2023-03-20T10:25:22.712 INFO:teuthology.orchestra.run.smithi072.stdout: "LUKEWARM", 2023-03-20T10:25:22.712 INFO:teuthology.orchestra.run.smithi072.stdout: "STANDARD" 2023-03-20T10:25:22.712 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2023-03-20T10:25:22.713 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:22.713 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:22.713 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-03-20T10:25:22.713 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2023-03-20T10:25:22.713 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2023-03-20T10:25:24.440 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2023-03-20T10:25:24.441 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2023-03-20T10:25:24.497 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:25:25.944 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:25:25.945 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:24.508+0000 7efc4b5f6780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:25:27.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:24.508+0000 7efc4b5f6780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:25:27.450 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:24.512+0000 7efc2cd6b700 20 reqs_thread_entry: start 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout: "id": "a09596be-35b9-426c-8b99-1da61aeeb17e", 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default", 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout: "domain_root": "default.rgw.meta:root", 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout: "control_pool": "default.rgw.control", 2023-03-20T10:25:27.451 INFO:teuthology.orchestra.run.smithi072.stdout: "gc_pool": "default.rgw.log:gc", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "lc_pool": "default.rgw.log:lc", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "log_pool": "default.rgw.log", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "roles_pool": "default.rgw.meta:roles", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "otp_pool": "default.rgw.otp", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "system_key": { 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "access_key": "", 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "secret_key": "" 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "placement_pools": [ 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-20T10:25:27.452 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "default-placement", 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "val": { 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "index_pool": "default.rgw.buckets.index", 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_classes": { 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "LUKEWARM": { 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "data_pool": "default.rgw.buckets.data.lukewarm" 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "STANDARD": { 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "data_pool": "default.rgw.buckets.data" 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "index_type": 0, 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "inline_data": true 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "realm_id": "", 2023-03-20T10:25:27.453 INFO:teuthology.orchestra.run.smithi072.stdout: "notif_pool": "default.rgw.log:notif" 2023-03-20T10:25:27.454 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-03-20T10:25:27.454 DEBUG:tasks.util.rgw: json result: {'id': 'a09596be-35b9-426c-8b99-1da61aeeb17e', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-03-20T10:25:27.454 INFO:tasks.rgw:Configuring storage class type = FROZEN 2023-03-20T10:25:27.454 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2023-03-20T10:25:27.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2023-03-20T10:25:27.454 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2023-03-20T10:25:27.513 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:25:29.677 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:25:29.681 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:27.524+0000 7fe4fa68b780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:25:29.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:27.524+0000 7fe4fa68b780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:25:29.682 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:27.528+0000 7fe4dbe00700 20 reqs_thread_entry: start 2023-03-20T10:25:29.682 INFO:teuthology.orchestra.run.smithi072.stdout:[ 2023-03-20T10:25:29.682 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-20T10:25:29.682 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "default-placement", 2023-03-20T10:25:29.683 INFO:teuthology.orchestra.run.smithi072.stdout: "val": { 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default-placement", 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "tags": [], 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_classes": [ 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "FROZEN", 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "LUKEWARM", 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: "STANDARD" 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:30.160 INFO:teuthology.orchestra.run.smithi072.stdout:] 2023-03-20T10:25:31.666 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2023-03-20T10:25:31.666 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2023-03-20T10:25:31.666 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2023-03-20T10:25:31.666 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2023-03-20T10:25:31.725 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:25:34.259 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:25:34.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:31.736+0000 7f4681a12780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:25:34.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:31.736+0000 7f4681a12780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:25:34.260 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:25:31.740+0000 7f4663187700 20 reqs_thread_entry: start 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "id": "a09596be-35b9-426c-8b99-1da61aeeb17e", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "name": "default", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "domain_root": "default.rgw.meta:root", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "control_pool": "default.rgw.control", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "gc_pool": "default.rgw.log:gc", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "lc_pool": "default.rgw.log:lc", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "log_pool": "default.rgw.log", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "intent_log_pool": "default.rgw.log:intent", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "usage_log_pool": "default.rgw.log:usage", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "roles_pool": "default.rgw.meta:roles", 2023-03-20T10:25:34.261 INFO:teuthology.orchestra.run.smithi072.stdout: "reshard_pool": "default.rgw.log:reshard", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "user_email_pool": "default.rgw.meta:users.email", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "otp_pool": "default.rgw.otp", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "system_key": { 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "access_key": "", 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: "secret_key": "" 2023-03-20T10:25:34.262 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "placement_pools": [ 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "default-placement", 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "val": { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "index_pool": "default.rgw.buckets.index", 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "storage_classes": { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "FROZEN": { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "data_pool": "default.rgw.buckets.data.frozen" 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "LUKEWARM": { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "data_pool": "default.rgw.buckets.data.lukewarm" 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "STANDARD": { 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "data_pool": "default.rgw.buckets.data" 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:25:34.263 INFO:teuthology.orchestra.run.smithi072.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: "index_type": 0, 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: "inline_data": true 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: "realm_id": "", 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout: "notif_pool": "default.rgw.log:notif" 2023-03-20T10:25:34.264 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-03-20T10:25:36.385 DEBUG:tasks.util.rgw: json result: {'id': 'a09596be-35b9-426c-8b99-1da61aeeb17e', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2023-03-20T10:25:40.521 INFO:tasks.rgw:Starting rgw... 2023-03-20T10:25:42.048 INFO:tasks.rgw:rgw client.0 config is {} 2023-03-20T10:25:42.048 INFO:tasks.rgw:Using beast as radosgw frontend 2023-03-20T10:25:42.048 INFO:tasks.rgw.client.0:Restarting daemon 2023-03-20T10:25:42.049 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2023-03-20T10:25:42.052 INFO:tasks.rgw.client.0:Started 2023-03-20T10:25:43.908 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi072.front.sepia.ceph.com:80/ 2023-03-20T10:25:43.908 DEBUG:teuthology.orchestra.run.smithi072:> curl http://smithi072.front.sepia.ceph.com:80/ 2023-03-20T10:25:43.945 INFO:teuthology.orchestra.run.smithi072.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2023-03-20T10:25:45.128 INFO:teuthology.orchestra.run.smithi072.stderr: Dload Upload Total Spent Left Speed 2023-03-20T10:25:45.939 INFO:teuthology.orchestra.run.smithi072.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 208k 2023-03-20T10:25:47.163 INFO:teuthology.orchestra.run.smithi072.stdout:anonymous 2023-03-20T10:25:47.163 INFO:teuthology.run_tasks:Running task rabbitmq... 2023-03-20T10:25:48.224 DEBUG:tasks.rabbitmq:RabbitMQ config is {'client.0': None} 2023-03-20T10:25:50.030 INFO:tasks.rabbitmq:Installing RabbitMQ... 2023-03-20T10:25:50.031 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install epel-release 2023-03-20T10:25:50.771 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:17:52 ago on Mon 20 Mar 2023 10:07:58 AM UTC. 2023-03-20T10:25:52.365 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:25:52.365 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:25:52.365 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout: epel-release noarch 8-18.el8 epel 24 k 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout: epel-next-release noarch 8-18.el8 epel 11 k 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:52.366 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:25:52.367 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:25:52.367 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2023-03-20T10:25:52.367 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:52.367 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 35 k 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 38 k 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): epel-next-release-8-18.el8.noarch.rpm 78 kB/s | 11 kB 00:00 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): epel-release-8-18.el8.noarch.rpm 142 kB/s | 24 kB 00:00 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:Total 203 kB/s | 35 kB 00:00 2023-03-20T10:25:52.898 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:25:52.899 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : epel-release-8-18.el8.noarch 1/2 2023-03-20T10:25:53.772 INFO:teuthology.orchestra.run.smithi072.stdout:warning: /etc/yum.repos.d/epel-testing.repo created as /etc/yum.repos.d/epel-testing.repo.rpmnew 2023-03-20T10:25:53.772 INFO:teuthology.orchestra.run.smithi072.stdout:warning: /etc/yum.repos.d/epel.repo created as /etc/yum.repos.d/epel.repo.rpmnew 2023-03-20T10:25:53.772 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: epel-release-8-18.el8.noarch 1/2 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout:Many EPEL packages require the CodeReady Builder (CRB) repository. 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout:It is recommended that you run /usr/bin/crb enable to enable the CRB repository. 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : epel-next-release-8-18.el8.noarch 2/2 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: epel-next-release-8-18.el8.noarch 2/2 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : epel-next-release-8-18.el8.noarch 1/2 2023-03-20T10:25:53.774 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : epel-release-8-18.el8.noarch 2/2 2023-03-20T10:25:53.775 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:53.775 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:25:53.775 INFO:teuthology.orchestra.run.smithi072.stdout: epel-next-release-8-18.el8.noarch epel-release-8-18.el8.noarch 2023-03-20T10:25:53.775 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:25:55.105 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:25:55.106 DEBUG:teuthology.orchestra.run.smithi072:> curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash 2023-03-20T10:25:55.546 INFO:teuthology.orchestra.run.smithi072.stdout:Detected operating system as centos/8. 2023-03-20T10:25:57.810 INFO:teuthology.orchestra.run.smithi072.stdout:Checking for curl... 2023-03-20T10:25:57.810 INFO:teuthology.orchestra.run.smithi072.stdout:Detected curl... 2023-03-20T10:25:57.810 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading repository file: https://packagecloud.io/install/repositories/rabbitmq/erlang/config_file.repo?os=centos&dist=8&source=script 2023-03-20T10:25:57.810 INFO:teuthology.orchestra.run.smithi072.stdout:done. 2023-03-20T10:25:57.811 INFO:teuthology.orchestra.run.smithi072.stdout:Installing yum-utils... 2023-03-20T10:25:57.811 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 8 - Next - 295 kB/s | 179 kB 00:00 2023-03-20T10:25:58.894 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_erlang-source 589 B/s | 819 B 00:01 2023-03-20T10:26:00.226 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_erlang-source 11 kB/s | 3.8 kB 00:00 2023-03-20T10:26:00.226 INFO:teuthology.orchestra.run.smithi072.stderr:Importing GPG key 0xDF309A0B: 2023-03-20T10:26:00.226 INFO:teuthology.orchestra.run.smithi072.stderr: Userid : "https://packagecloud.io/rabbitmq/erlang (https://packagecloud.io/docs#gpg_signing) " 2023-03-20T10:26:00.878 INFO:teuthology.orchestra.run.smithi072.stderr: Fingerprint: 2EBD E413 D3CE 5D35 BCD1 5B7C 71C6 3471 DF30 9A0B 2023-03-20T10:26:00.878 INFO:teuthology.orchestra.run.smithi072.stderr: From : https://packagecloud.io/rabbitmq/erlang/gpgkey 2023-03-20T10:26:01.114 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_erlang-source 166 B/s | 296 B 00:01 2023-03-20T10:26:02.853 INFO:teuthology.orchestra.run.smithi072.stdout:Package yum-utils-4.0.21-18.el8.noarch is already installed. 2023-03-20T10:26:05.171 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:26:05.171 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:26:05.171 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:26:05.171 INFO:teuthology.orchestra.run.smithi072.stdout:Generating yum cache for rabbitmq_erlang... 2023-03-20T10:26:05.822 INFO:teuthology.orchestra.run.smithi072.stderr:Importing GPG key 0xDF309A0B: 2023-03-20T10:26:06.663 INFO:teuthology.orchestra.run.smithi072.stderr: Userid : "https://packagecloud.io/rabbitmq/erlang (https://packagecloud.io/docs#gpg_signing) " 2023-03-20T10:26:08.871 INFO:teuthology.orchestra.run.smithi072.stderr: Fingerprint: 2EBD E413 D3CE 5D35 BCD1 5B7C 71C6 3471 DF30 9A0B 2023-03-20T10:26:08.871 INFO:teuthology.orchestra.run.smithi072.stderr: From : https://packagecloud.io/rabbitmq/erlang/gpgkey 2023-03-20T10:26:08.872 INFO:teuthology.orchestra.run.smithi072.stdout:Generating yum cache for rabbitmq_erlang-source... 2023-03-20T10:26:10.938 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:10.938 INFO:teuthology.orchestra.run.smithi072.stdout:The repository is setup! You can now install packages. 2023-03-20T10:26:10.938 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install erlang 2023-03-20T10:26:11.753 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:02 ago on Mon 20 Mar 2023 10:26:09 AM UTC. 2023-03-20T10:26:13.045 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:26:15.552 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:15.552 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout: erlang x86_64 25.3-1.el8 rabbitmq_erlang 20 M 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:26:15.553 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:17.775 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 20 M 2023-03-20T10:26:17.775 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 36 M 2023-03-20T10:26:17.775 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:erlang-25.3-1.el8.x86_64.rpm 20 MB/s | 20 MB 00:01 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Total 20 MB/s | 20 MB 00:01 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:26:19.011 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:26:19.012 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:26:19.013 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : erlang-25.3-1.el8.x86_64 1/1 2023-03-20T10:26:19.013 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: erlang-25.3-1.el8.x86_64 1/1 2023-03-20T10:26:19.013 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : erlang-25.3-1.el8.x86_64 1/1 2023-03-20T10:26:19.013 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:20.055 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:26:20.055 INFO:teuthology.orchestra.run.smithi072.stdout: erlang-25.3-1.el8.x86_64 2023-03-20T10:26:20.055 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:20.055 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:26:20.056 DEBUG:teuthology.orchestra.run.smithi072:> curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash 2023-03-20T10:26:20.481 INFO:teuthology.orchestra.run.smithi072.stdout:Detected operating system as centos/8. 2023-03-20T10:26:20.927 INFO:teuthology.orchestra.run.smithi072.stdout:Checking for curl... 2023-03-20T10:26:20.927 INFO:teuthology.orchestra.run.smithi072.stdout:Detected curl... 2023-03-20T10:26:20.928 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading repository file: https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/config_file.repo?os=centos&dist=8&source=script 2023-03-20T10:26:21.146 INFO:teuthology.orchestra.run.smithi072.stdout:done. 2023-03-20T10:26:22.013 INFO:teuthology.orchestra.run.smithi072.stdout:Installing yum-utils... 2023-03-20T10:26:23.204 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_rabbitmq-server-source 636 B/s | 819 B 00:01 2023-03-20T10:26:23.837 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_rabbitmq-server-source 11 kB/s | 3.9 kB 00:00 2023-03-20T10:26:23.838 INFO:teuthology.orchestra.run.smithi072.stderr:Importing GPG key 0x4D206F89: 2023-03-20T10:26:23.838 INFO:teuthology.orchestra.run.smithi072.stderr: Userid : "https://packagecloud.io/rabbitmq/rabbitmq-server (https://packagecloud.io/docs#gpg_signing) " 2023-03-20T10:26:24.484 INFO:teuthology.orchestra.run.smithi072.stderr: Fingerprint: 8C69 5B02 19AF DEB0 4A05 8ED8 F4E7 8920 4D20 6F89 2023-03-20T10:26:24.484 INFO:teuthology.orchestra.run.smithi072.stderr: From : https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey 2023-03-20T10:26:25.378 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq_rabbitmq-server-source 168 B/s | 296 B 00:01 2023-03-20T10:26:27.016 INFO:teuthology.orchestra.run.smithi072.stdout:Package yum-utils-4.0.21-18.el8.noarch is already installed. 2023-03-20T10:26:28.944 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:26:28.944 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2023-03-20T10:26:28.944 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:26:30.597 INFO:teuthology.orchestra.run.smithi072.stdout:Generating yum cache for rabbitmq_rabbitmq-server... 2023-03-20T10:26:30.598 INFO:teuthology.orchestra.run.smithi072.stderr:Importing GPG key 0x4D206F89: 2023-03-20T10:26:30.598 INFO:teuthology.orchestra.run.smithi072.stderr: Userid : "https://packagecloud.io/rabbitmq/rabbitmq-server (https://packagecloud.io/docs#gpg_signing) " 2023-03-20T10:26:30.598 INFO:teuthology.orchestra.run.smithi072.stderr: Fingerprint: 8C69 5B02 19AF DEB0 4A05 8ED8 F4E7 8920 4D20 6F89 2023-03-20T10:26:30.598 INFO:teuthology.orchestra.run.smithi072.stderr: From : https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey 2023-03-20T10:26:32.652 INFO:teuthology.orchestra.run.smithi072.stdout:Generating yum cache for rabbitmq_rabbitmq-server-source... 2023-03-20T10:26:37.134 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:37.134 INFO:teuthology.orchestra.run.smithi072.stdout:The repository is setup! You can now install packages. 2023-03-20T10:26:37.134 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rabbitmq-server 2023-03-20T10:26:37.993 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:04 ago on Mon 20 Mar 2023 10:26:33 AM UTC. 2023-03-20T10:26:39.834 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:26:39.834 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:39.834 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:26:39.834 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:41.041 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: rabbitmq-server noarch 3.11.10-1.el8 rabbitmq_rabbitmq-server 17 M 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 17 M 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 22 M 2023-03-20T10:26:42.257 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:26:42.259 INFO:teuthology.orchestra.run.smithi072.stdout:rabbitmq-server-3.11.10-1.el8.noarch.rpm 16 MB/s | 17 MB 00:01 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Total 15 MB/s | 17 MB 00:01 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:26:42.260 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:26:43.037 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rabbitmq-server-3.11.10-1.el8.noarch 1/1 2023-03-20T10:26:43.711 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rabbitmq-server-3.11.10-1.el8.noarch 1/1 2023-03-20T10:26:44.879 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rabbitmq-server-3.11.10-1.el8.noarch 1/1 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout:[/usr/lib/tmpfiles.d/rabbitmq-server.conf:1] Line references path below legacy directory /var/run/, updating /var/run/rabbitmq → /run/rabbitmq; please update the tmpfiles.d/ drop-in file accordingly. 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rabbitmq-server-3.11.10-1.el8.noarch 1/1 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout: rabbitmq-server-3.11.10-1.el8.noarch 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:26:44.880 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:26:44.881 INFO:tasks.rabbitmq:Bringing up Daemon and RabbitMQ service... 2023-03-20T10:26:44.882 DEBUG:teuthology.orchestra.run.smithi072:> sudo chkconfig rabbitmq-server on 2023-03-20T10:26:44.931 INFO:teuthology.orchestra.run.smithi072.stderr:Note: Forwarding request to 'systemctl enable rabbitmq-server.service'. 2023-03-20T10:26:46.777 INFO:teuthology.orchestra.run.smithi072.stderr:Created symlink /etc/systemd/system/multi-user.target.wants/rabbitmq-server.service → /usr/lib/systemd/system/rabbitmq-server.service. 2023-03-20T10:26:46.777 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/service rabbitmq-server start 2023-03-20T10:26:46.821 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl start rabbitmq-server.service 2023-03-20T10:26:53.328 INFO:teuthology.run_tasks:Running task notification-tests... 2023-03-20T10:26:53.351 DEBUG:tasks.notification_tests:Notifications config is {'client.0': {'extra_attr': ['amqp_test'], 'rgw_server': 'client.0'}} 2023-03-20T10:26:55.295 INFO:tasks.notification_tests:Downloading bucket-notifications-tests... 2023-03-20T10:26:55.296 INFO:tasks.notification_tests:Using branch reef from https://github.com/ceph/ceph-ci.git for bucket notifications tests 2023-03-20T10:26:55.296 DEBUG:teuthology.orchestra.run.smithi072:> git clone -b reef https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/ceph 2023-03-20T10:26:57.637 INFO:teuthology.orchestra.run.smithi072.stderr:Cloning into '/home/ubuntu/cephtest/ceph'... 2023-03-20T10:27:41.141 INFO:tasks.notification_tests:Pre-processing... 2023-03-20T10:27:42.408 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p /home/ubuntu/.aws/models/s3/2006-03-01/ 2023-03-20T10:27:42.428 DEBUG:teuthology.orchestra.run.smithi072:> cd /home/ubuntu/.aws/models/s3/2006-03-01/ && cp /home/ubuntu/cephtest/ceph/examples/rgw/boto3/service-2.sdk-extras.json service-2.sdk-extras.json 2023-03-20T10:27:44.257 INFO:tasks.notification_tests:Creating rgw user... 2023-03-20T10:27:47.551 DEBUG:tasks.notification_tests:Creating user foo.client.0 on client.0 2023-03-20T10:27:47.552 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key YDFTONNSXZETVCOKUMGC --secret U/TmfLDTswa7BX4MvJ0hldaB5MkGEQ1CctPS8ViJ7FUnzINwWfbLNw== --cluster ceph 2023-03-20T10:27:47.609 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setuser ceph since I am not root 2023-03-20T10:27:48.874 INFO:teuthology.orchestra.run.smithi072.stderr:ignoring --setgroup ceph since I am not root 2023-03-20T10:27:48.878 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.612+0000 7fa4213a2780 -1 asok(0x5633c8b9c000) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.0.119627.asok': (13) Permission denied 2023-03-20T10:27:51.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.620+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:51.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:51.979 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 realm 2023-03-20T10:27:53.730 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.622+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.623+0000 7fa4213a2780 20 rados_obj.operate() r=0 bl.length=46 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.623+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.624+0000 7fa4213a2780 20 rados_obj.operate() r=0 bl.length=1004 2023-03-20T10:27:53.731 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.624+0000 7fa4213a2780 20 searching for the correct realm 2023-03-20T10:27:55.217 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.636+0000 7fa4213a2780 20 RGWRados::pool_iterate: got zone_info.a09596be-35b9-426c-8b99-1da61aeeb17e 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.636+0000 7fa4213a2780 20 RGWRados::pool_iterate: got zone_names.default 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.636+0000 7fa4213a2780 20 RGWRados::pool_iterate: got zonegroups_names.default 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.636+0000 7fa4213a2780 20 RGWRados::pool_iterate: got zonegroup_info.2d432b72-35d2-4754-a1e2-8b1c802e34d5 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.636+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados_obj.operate() r=0 bl.length=46 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.790 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados_obj.operate() r=0 bl.length=432 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 zone default found 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 4 Realm: () 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 4 ZoneGroup: default (2d432b72-35d2-4754-a1e2-8b1c802e34d5) 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 4 Zone: default (a09596be-35b9-426c-8b99-1da61aeeb17e) 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 10 cannot find current period zonegroup using local zonegroup configuration 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 zonegroup default 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.637+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 started sync module instance, tier type = 2023-03-20T10:27:55.791 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.638+0000 7fa4213a2780 20 started zone id=a09596be-35b9-426c-8b99-1da61aeeb17e (name=default) with tier type = 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=0 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=1 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=2 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=3 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=4 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=5 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=6 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 20 add_watcher() i=7 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.648+0000 7fa4213a2780 2 all 8 watchers are set, enabling cache 2023-03-20T10:27:56.937 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.651+0000 7fa406b1f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-03-20T10:27:57.545 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.651+0000 7fa4213a2780 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.651+0000 7fa4213a2780 20 check_secure_mon_conn(): mode 1 is insecure 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.656+0000 7fa4213a2780 5 note: GC not initialized 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa402316700 20 reqs_thread_entry: start 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 20 init_complete bucket index max shards: 11 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 10 rgw notify: Started notification manager with: 1 workers 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa400312700 20 rgw notify: INFO: next queues processing will happen at: Mon Mar 20 10:28:17 2023 2023-03-20T10:27:59.046 INFO:teuthology.orchestra.run.smithi072.stderr: 2023-03-20T10:27:59.536 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa3ffaf0700 20 reqs_thread_entry: start 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 10 cache get: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC : miss 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.657+0000 7fa4213a2780 20 rados->read ofs=0 len=0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.658+0000 7fa4213a2780 20 rados_obj.operate() r=-2 bl.length=0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.658+0000 7fa4213a2780 10 cache put: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC info.flags=0x0 2023-03-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.658+0000 7fa4213a2780 10 adding default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC to cache LRU end 2023-03-20T10:28:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.658+0000 7fa4213a2780 10 cache get: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC : hit (negative entry) 2023-03-20T10:28:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.658+0000 7fa4213a2780 10 cache get: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC : hit (negative entry) 2023-03-20T10:28:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.659+0000 7fa4213a2780 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2023-03-20T10:28:01.513 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.659+0000 7fa4213a2780 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-03-20T10:28:02.318 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.659+0000 7fa4213a2780 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.660+0000 7fa41a346700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 193273528320 cookie 94780410926720 notifier 4518 bl.length()=594 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.660+0000 7fa41a346700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.660+0000 7fa41a346700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.661+0000 7fa4213a2780 10 cache put: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC info.flags=0x7 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.661+0000 7fa4213a2780 10 moving default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC to cache LRU end 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.661+0000 7fa4213a2780 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:YDFTONNSXZETVCOKUMGC, ofs0, ns] 2023-03-20T10:28:02.319 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.662+0000 7fa41a346700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 193273528320 cookie 94780410932480 notifier 4518 bl.length()=186 2023-03-20T10:28:03.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.662+0000 7fa41a346700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC info.flags=0x7 2023-03-20T10:28:03.561 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.662+0000 7fa41a346700 10 rgw watcher librados: moving default.rgw.meta+users.keys+YDFTONNSXZETVCOKUMGC to cache LRU end 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout: "user_id": "foo.client.0", 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout: "display_name": "Mr. foo.client.0", 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout: "email": "", 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout: "suspended": 0, 2023-03-20T10:28:03.562 INFO:teuthology.orchestra.run.smithi072.stdout: "max_buckets": 1000, 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "subusers": [], 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "keys": [ 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: { 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "user": "foo.client.0", 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "access_key": "YDFTONNSXZETVCOKUMGC", 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "secret_key": "U/TmfLDTswa7BX4MvJ0hldaB5MkGEQ1CctPS8ViJ7FUnzINwWfbLNw==" 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: } 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "swift_keys": [], 2023-03-20T10:28:03.563 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": [], 2023-03-20T10:28:05.850 INFO:teuthology.orchestra.run.smithi072.stdout: "op_mask": "read, write, delete", 2023-03-20T10:28:05.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_placement": "", 2023-03-20T10:28:05.850 INFO:teuthology.orchestra.run.smithi072.stdout: "default_storage_class": "", 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "placement_tags": [], 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "bucket_quota": { 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": false, 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "check_on_raw": false, 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "max_size": -1, 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "max_size_kb": 0, 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "max_objects": -1 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:28:05.851 INFO:teuthology.orchestra.run.smithi072.stdout: "user_quota": { 2023-03-20T10:28:06.689 INFO:teuthology.orchestra.run.smithi072.stdout: "enabled": false, 2023-03-20T10:28:06.689 INFO:teuthology.orchestra.run.smithi072.stdout: "check_on_raw": false, 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "max_size": -1, 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "max_size_kb": 0, 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "max_objects": -1 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "temp_url_keys": [], 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "type": "rgw", 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: "mfa_ids": [] 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout:} 2023-03-20T10:28:08.367 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:28:08.369 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.663+0000 7fa406b1f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2023-03-20T10:28:12.661 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.665+0000 7fa4213a2780 20 remove_watcher() i=0 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.665+0000 7fa4213a2780 2 removed watcher, disabling cache 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.666+0000 7fa4213a2780 20 remove_watcher() i=1 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.667+0000 7fa4213a2780 20 remove_watcher() i=2 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.669+0000 7fa4213a2780 20 remove_watcher() i=3 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.670+0000 7fa4213a2780 20 remove_watcher() i=4 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.671+0000 7fa4213a2780 20 remove_watcher() i=5 2023-03-20T10:28:12.662 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.672+0000 7fa4213a2780 20 remove_watcher() i=6 2023-03-20T10:28:12.663 INFO:teuthology.orchestra.run.smithi072.stderr:2023-03-20T10:27:47.673+0000 7fa4213a2780 20 remove_watcher() i=7 2023-03-20T10:28:12.663 INFO:tasks.notification_tests:Configuring bucket-notifications-tests... 2023-03-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2023-03-20T10:28:15.355 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf 2023-03-20T10:28:15.378 DEBUG:teuthology.orchestra.run.smithi072:> cd /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification && ./bootstrap 2023-03-20T10:28:25.787 INFO:teuthology.orchestra.run.smithi072.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2023-03-20T10:28:28.861 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:53 ago on Mon 20 Mar 2023 10:26:33 AM UTC. 2023-03-20T10:28:28.861 INFO:teuthology.orchestra.run.smithi072.stdout:Package python36-devel-3.6.8-38.module_el8.5.0+895+a459eca8.x86_64 is already installed. 2023-03-20T10:28:28.861 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2023-03-20T10:28:28.861 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:28:28.861 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2023-03-20T10:28:28.862 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:28:28.862 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2023-03-20T10:28:28.862 INFO:teuthology.orchestra.run.smithi072.stdout: libevent-devel x86_64 2.1.8-5.el8 CentOS-AppStream 104 k 2023-03-20T10:28:31.790 INFO:teuthology.orchestra.run.smithi072.stdout: libxml2-devel x86_64 2.9.7-16.el8 CentOS-AppStream 1.0 M 2023-03-20T10:28:34.030 INFO:teuthology.orchestra.run.smithi072.stdout: libxslt-devel x86_64 1.1.32-6.el8 CentOS-AppStream 323 k 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pika noarch 1.0.1-3.el8.1 epel 258 k 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout: python3-xmltodict noarch 0.12.0-6.el8 epel 24 k 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-21.el8 CentOS-BaseOS 58 k 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout: libxml2 x86_64 2.9.7-16.el8 CentOS-BaseOS 696 k 2023-03-20T10:28:34.031 INFO:teuthology.orchestra.run.smithi072.stdout: python3-libxml2 x86_64 2.9.7-16.el8 CentOS-BaseOS 237 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: zlib x86_64 1.2.11-21.el8 CentOS-BaseOS 103 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: cmake-filesystem x86_64 3.20.2-5.el8 CentOS-AppStream 45 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: libgcrypt-devel x86_64 1.8.5-7.el8 CentOS-BaseOS 150 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: libgpg-error-devel x86_64 1.31-1.el8 CentOS-BaseOS 64 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: python3-Automat noarch 0.7.0-3.1.el8 epel 66 k 2023-03-20T10:28:35.099 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 17.4.0-6.el8 CentOS-AppStream 53 k 2023-03-20T10:28:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: python3-constantly noarch 15.1.0-4.el8 epel 19 k 2023-03-20T10:28:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: python3-hamcrest noarch 1.9.0-8.el8 epel 75 k 2023-03-20T10:28:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: python3-hyperlink noarch 19.0.0-1.el8.1 epel 75 k 2023-03-20T10:28:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: python3-incremental noarch 17.5.0-5.el8.1 epel 33 k 2023-03-20T10:28:35.100 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycurl x86_64 7.43.0.2-4.el8 CentOS-AppStream 227 k 2023-03-20T10:28:36.735 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zope-event noarch 4.2.0-12.el8 epel 210 k 2023-03-20T10:28:36.735 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zope-interface x86_64 4.6.0-1.el8 epel 158 k 2023-03-20T10:28:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: xz-devel x86_64 5.2.4-4.el8 CentOS-BaseOS 63 k 2023-03-20T10:28:37.542 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2023-03-20T10:28:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: python3-service-identity noarch 18.1.0-2.el8 epel 28 k 2023-03-20T10:28:37.542 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tornado x86_64 6.0.2-1.el8 epel 722 k 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout: python3-twisted x86_64 19.10.0-2.el8 epel 6.7 M 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout:Install 22 Packages 2023-03-20T10:28:38.987 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 3 Packages 2023-03-20T10:28:40.017 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:28:42.856 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 11 M 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(1/25): cmake-filesystem-3.20.2-5.el8.x86_64.rp 382 kB/s | 45 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(2/25): libevent-devel-2.1.8-5.el8.x86_64.rpm 541 kB/s | 104 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(3/25): libxslt-devel-1.1.32-6.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(4/25): libxml2-devel-2.9.7-16.el8.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(5/25): python3-attrs-17.4.0-6.el8.noarch.rpm 522 kB/s | 53 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(6/25): python3-pycurl-7.43.0.2-4.el8.x86_64.rp 2.4 MB/s | 227 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(7/25): libgcrypt-devel-1.8.5-7.el8.x86_64.rpm 1.7 MB/s | 150 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(8/25): libgpg-error-devel-1.31-1.el8.x86_64.rp 741 kB/s | 64 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(9/25): xz-devel-5.2.4-4.el8.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-03-20T10:28:42.857 INFO:teuthology.orchestra.run.smithi072.stdout:(10/25): zlib-devel-1.2.11-21.el8.x86_64.rpm 759 kB/s | 58 kB 00:00 2023-03-20T10:28:42.862 INFO:teuthology.orchestra.run.smithi072.stdout:(11/25): python3-constantly-15.1.0-4.el8.noarch 7.1 kB/s | 19 kB 00:02 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(12/25): python3-Automat-0.7.0-3.1.el8.noarch.r 24 kB/s | 66 kB 00:02 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(13/25): python3-hamcrest-1.9.0-8.el8.noarch.rp 27 kB/s | 75 kB 00:02 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(14/25): python3-hyperlink-19.0.0-1.el8.1.noarc 892 kB/s | 75 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(15/25): python3-incremental-17.5.0-5.el8.1.noa 427 kB/s | 33 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(16/25): python3-service-identity-18.1.0-2.el8. 477 kB/s | 28 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(17/25): python3-pika-1.0.1-3.el8.1.noarch.rpm 1.3 MB/s | 258 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(18/25): python3-xmltodict-0.12.0-6.el8.noarch. 243 kB/s | 24 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(19/25): python3-tornado-6.0.2-1.el8.x86_64.rpm 2.0 MB/s | 722 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(20/25): python3-zope-interface-4.6.0-1.el8.x86 1.2 MB/s | 158 kB 00:00 2023-03-20T10:28:43.638 INFO:teuthology.orchestra.run.smithi072.stdout:(21/25): python3-zope-event-4.2.0-12.el8.noarch 694 kB/s | 210 kB 00:00 2023-03-20T10:28:44.297 INFO:teuthology.orchestra.run.smithi072.stdout:(22/25): libxml2-2.9.7-16.el8.x86_64.rpm 6.7 MB/s | 696 kB 00:00 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:(23/25): python3-libxml2-2.9.7-16.el8.x86_64.rp 3.1 MB/s | 237 kB 00:00 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:(24/25): zlib-1.2.11-21.el8.x86_64.rpm 1.7 MB/s | 103 kB 00:00 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:(25/25): python3-twisted-19.10.0-2.el8.x86_64.r 4.0 MB/s | 6.7 MB 00:01 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Total 2.3 MB/s | 11 MB 00:04 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2023-03-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2023-03-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : zlib-1.2.11-21.el8.x86_64 1/28 2023-03-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-17.4.0-6.el8.noarch 2/28 2023-03-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : libxml2-2.9.7-16.el8.x86_64 3/28 2023-03-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgpg-error-devel-1.31-1.el8.x86_64 4/28 2023-03-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgpg-error-devel-1.31-1.el8.x86_64 4/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgcrypt-devel-1.8.5-7.el8.x86_64 5/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libgcrypt-devel-1.8.5-7.el8.x86_64 5/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-Automat-0.7.0-3.1.el8.noarch 6/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-service-identity-18.1.0-2.el8.noarch 7/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 8/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zope-event-4.2.0-12.el8.noarch 9/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zope-interface-4.6.0-1.el8.x86_64 10/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-incremental-17.5.0-5.el8.1.noarch 11/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-hyperlink-19.0.0-1.el8.1.noarch 12/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-hamcrest-1.9.0-8.el8.noarch 13/28 2023-03-20T10:28:46.030 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-constantly-15.1.0-4.el8.noarch 14/28 2023-03-20T10:28:46.031 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-twisted-19.10.0-2.el8.x86_64 15/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xz-devel-5.2.4-4.el8.x86_64 16/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycurl-7.43.0.2-4.el8.x86_64 17/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tornado-6.0.2-1.el8.x86_64 18/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cmake-filesystem-3.20.2-5.el8.x86_64 19/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libxml2-devel-2.9.7-16.el8.x86_64 20/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libxslt-devel-1.1.32-6.el8.x86_64 21/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pika-1.0.1-3.el8.1.noarch 22/28 2023-03-20T10:28:47.392 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : python3-libxml2-2.9.7-16.el8.x86_64 23/28 2023-03-20T10:28:47.395 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-xmltodict-0.12.0-6.el8.noarch 24/28 2023-03-20T10:28:48.615 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libevent-devel-2.1.8-5.el8.x86_64 25/28 2023-03-20T10:28:48.615 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : python3-libxml2-2.9.7-14.el8.x86_64 26/28 2023-03-20T10:28:48.615 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : libxml2-2.9.7-14.el8.x86_64 27/28 2023-03-20T10:28:48.615 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : zlib-1.2.11-19.el8.x86_64 28/28 2023-03-20T10:28:53.651 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: zlib-1.2.11-19.el8.x86_64 28/28 2023-03-20T10:28:53.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cmake-filesystem-3.20.2-5.el8.x86_64 1/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libevent-devel-2.1.8-5.el8.x86_64 2/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libxml2-devel-2.9.7-16.el8.x86_64 3/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libxslt-devel-1.1.32-6.el8.x86_64 4/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-17.4.0-6.el8.noarch 5/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycurl-7.43.0.2-4.el8.x86_64 6/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgcrypt-devel-1.8.5-7.el8.x86_64 7/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgpg-error-devel-1.31-1.el8.x86_64 8/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xz-devel-5.2.4-4.el8.x86_64 9/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 10/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-Automat-0.7.0-3.1.el8.noarch 11/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-constantly-15.1.0-4.el8.noarch 12/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-hamcrest-1.9.0-8.el8.noarch 13/28 2023-03-20T10:28:53.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-hyperlink-19.0.0-1.el8.1.noarch 14/28 2023-03-20T10:28:54.843 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-incremental-17.5.0-5.el8.1.noarch 15/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pika-1.0.1-3.el8.1.noarch 16/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-service-identity-18.1.0-2.el8.noarch 17/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tornado-6.0.2-1.el8.x86_64 18/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-twisted-19.10.0-2.el8.x86_64 19/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-xmltodict-0.12.0-6.el8.noarch 20/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zope-event-4.2.0-12.el8.noarch 21/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zope-interface-4.6.0-1.el8.x86_64 22/28 2023-03-20T10:28:54.844 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libxml2-2.9.7-16.el8.x86_64 23/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libxml2-2.9.7-14.el8.x86_64 24/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-libxml2-2.9.7-16.el8.x86_64 25/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-libxml2-2.9.7-14.el8.x86_64 26/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-1.2.11-21.el8.x86_64 27/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-1.2.11-19.el8.x86_64 28/28 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:28:54.845 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: libxml2-2.9.7-16.el8.x86_64 python3-libxml2-2.9.7-16.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-1.2.11-21.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: cmake-filesystem-3.20.2-5.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: libevent-devel-2.1.8-5.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: libgcrypt-devel-1.8.5-7.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: libgpg-error-devel-1.31-1.el8.x86_64 2023-03-20T10:28:54.846 INFO:teuthology.orchestra.run.smithi072.stdout: libxml2-devel-2.9.7-16.el8.x86_64 2023-03-20T10:28:55.496 INFO:teuthology.orchestra.run.smithi072.stdout: libxslt-devel-1.1.32-6.el8.x86_64 2023-03-20T10:28:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-Automat-0.7.0-3.1.el8.noarch 2023-03-20T10:28:56.253 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-17.4.0-6.el8.noarch 2023-03-20T10:28:58.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-constantly-15.1.0-4.el8.noarch 2023-03-20T10:28:58.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-hamcrest-1.9.0-8.el8.noarch 2023-03-20T10:28:58.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-hyperlink-19.0.0-1.el8.1.noarch 2023-03-20T10:28:58.953 INFO:teuthology.orchestra.run.smithi072.stdout: python3-incremental-17.5.0-5.el8.1.noarch 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pika-1.0.1-3.el8.1.noarch 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycurl-7.43.0.2-4.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-service-identity-18.1.0-2.el8.noarch 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tornado-6.0.2-1.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-twisted-19.10.0-2.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-xmltodict-0.12.0-6.el8.noarch 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zope-event-4.2.0-12.el8.noarch 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zope-interface-4.6.0-1.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: xz-devel-5.2.4-4.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:00.960 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2023-03-20T10:29:00.962 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting pip 2023-03-20T10:29:00.962 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB) 2023-03-20T10:29:00.962 INFO:teuthology.orchestra.run.smithi072.stdout:Installing collected packages: pip 2023-03-20T10:29:00.962 INFO:teuthology.orchestra.run.smithi072.stdout: Found existing installation: pip 9.0.3 2023-03-20T10:29:05.080 INFO:teuthology.orchestra.run.smithi072.stdout: Uninstalling pip-9.0.3: 2023-03-20T10:29:05.080 INFO:teuthology.orchestra.run.smithi072.stdout: Successfully uninstalled pip-9.0.3 2023-03-20T10:29:05.080 INFO:teuthology.orchestra.run.smithi072.stdout:Successfully installed pip-21.3.1 2023-03-20T10:29:05.081 INFO:teuthology.orchestra.run.smithi072.stderr:You are using pip version 21.3.1, however version 23.0.1 is available. 2023-03-20T10:29:06.452 INFO:teuthology.orchestra.run.smithi072.stderr:You should consider upgrading via the 'pip install --upgrade pip' command. 2023-03-20T10:29:06.455 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: nose>=1.0.0 in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (1.3.7) 2023-03-20T10:29:06.455 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting boto>=2.6.0 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading boto-2.49.0-py2.py3-none-any.whl (1.4 MB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting boto3>=1.0.0 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading boto3-1.23.10-py3-none-any.whl (132 kB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting configparser>=5.0.0 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading configparser-5.2.0-py3-none-any.whl (19 kB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting kafka-python>=2.0.0 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading kafka_python-2.0.2-py2.py3-none-any.whl (246 kB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: pika in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (1.0.1) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting pika 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading pika-1.3.1-py3-none-any.whl (155 kB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting cloudevents 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading cloudevents-1.9.0-py3-none-any.whl (47 kB) 2023-03-20T10:29:06.456 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: xmltodict in /usr/lib/python3.6/site-packages (from -r requirements.txt (line 8)) (0.12.0) 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting xmltodict 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading xmltodict-0.13.0-py2.py3-none-any.whl (10.0 kB) 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting s3transfer<0.6.0,>=0.5.0 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting botocore<1.27.0,>=1.26.10 2023-03-20T10:29:06.457 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading botocore-1.26.10-py3-none-any.whl (8.8 MB) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.6/site-packages (from boto3>=1.0.0->-r requirements.txt (line 3)) (0.9.0) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting deprecation<3.0,>=2.0 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading deprecation-2.1.0-py2.py3-none-any.whl (11 kB) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.6/site-packages (from botocore<1.27.0,>=1.26.10->boto3>=1.0.0->-r requirements.txt (line 3)) (2.6.1) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting urllib3<1.27,>=1.25.4 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading urllib3-1.26.15-py2.py3-none-any.whl (140 kB) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Collecting packaging 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Downloading packaging-21.3-py3-none-any.whl (40 kB) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.27.0,>=1.26.10->boto3>=1.0.0->-r requirements.txt (line 3)) (1.11.0) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3.6/site-packages (from packaging->deprecation<3.0,>=2.0->cloudevents->-r requirements.txt (line 7)) (2.1.10) 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout:Installing collected packages: urllib3, packaging, botocore, s3transfer, deprecation, xmltodict, pika, kafka-python, configparser, cloudevents, boto3, boto 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Attempting uninstall: urllib3 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Found existing installation: urllib3 1.24.2 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Not uninstalling urllib3 at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Can't uninstall 'urllib3'. No files were found to uninstall. 2023-03-20T10:29:06.458 INFO:teuthology.orchestra.run.smithi072.stdout: Attempting uninstall: xmltodict 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Found existing installation: xmltodict 0.12.0 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Not uninstalling xmltodict at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Can't uninstall 'xmltodict'. No files were found to uninstall. 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Attempting uninstall: pika 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Found existing installation: pika 1.0.1 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Not uninstalling pika at /usr/lib/python3.6/site-packages, outside environment /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stdout: Can't uninstall 'pika'. No files were found to uninstall. 2023-03-20T10:29:06.459 INFO:teuthology.orchestra.run.smithi072.stderr:ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2023-03-20T10:29:06.851 INFO:teuthology.orchestra.run.smithi072.stderr:requests 2.20.0 requires urllib3<1.25,>=1.21.1, but you have urllib3 1.26.15 which is incompatible. 2023-03-20T10:29:06.855 INFO:teuthology.orchestra.run.smithi072.stdout:Successfully installed boto-2.49.0 boto3-1.23.10 botocore-1.26.10 cloudevents-1.9.0 configparser-5.2.0 deprecation-2.1.0 kafka-python-2.0.2 packaging-21.3 pika-1.3.1 s3transfer-0.5.2 urllib3-1.26.15 xmltodict-0.13.0 2023-03-20T10:29:07.954 INFO:teuthology.orchestra.run.smithi072.stdout:running develop 2023-03-20T10:29:09.506 INFO:teuthology.orchestra.run.smithi072.stdout:running egg_info 2023-03-20T10:29:09.506 INFO:teuthology.orchestra.run.smithi072.stdout:creating bn_tests.egg-info 2023-03-20T10:29:09.506 INFO:teuthology.orchestra.run.smithi072.stdout:writing bn_tests.egg-info/PKG-INFO 2023-03-20T10:29:09.506 INFO:teuthology.orchestra.run.smithi072.stdout:writing dependency_links to bn_tests.egg-info/dependency_links.txt 2023-03-20T10:29:09.506 INFO:teuthology.orchestra.run.smithi072.stdout:writing requirements to bn_tests.egg-info/requires.txt 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:writing top-level names to bn_tests.egg-info/top_level.txt 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:writing manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:reading manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:writing manifest file 'bn_tests.egg-info/SOURCES.txt' 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:running build_ext 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:Creating /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages/bn-tests.egg-link (link to .) 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:Adding bn-tests 0.0.1 to easy-install.pth file 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:Installed /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification 2023-03-20T10:29:09.507 INFO:teuthology.orchestra.run.smithi072.stdout:Processing dependencies for bn-tests==0.0.1 2023-03-20T10:29:10.127 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for boto3==1.23.10 2023-03-20T10:29:10.127 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: boto3 1.23.10 2023-03-20T10:29:10.127 INFO:teuthology.orchestra.run.smithi072.stdout:Adding boto3 1.23.10 to easy-install.pth file 2023-03-20T10:29:11.113 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.113 INFO:teuthology.orchestra.run.smithi072.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-03-20T10:29:11.113 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for boto==2.49.0 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: boto 2.49.0 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Adding boto 2.49.0 to easy-install.pth file 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for s3transfer==0.5.2 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: s3transfer 0.5.2 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Adding s3transfer 0.5.2 to easy-install.pth file 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for jmespath==0.9.0 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: jmespath 0.9.0 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Adding jmespath 0.9.0 to easy-install.pth file 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Using /usr/lib/python3.6/site-packages 2023-03-20T10:29:11.114 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for botocore==1.26.10 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: botocore 1.26.10 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Adding botocore 1.26.10 to easy-install.pth file 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for python-dateutil==2.6.1 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: python-dateutil 2.6.1 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Adding python-dateutil 2.6.1 to easy-install.pth file 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:11.115 INFO:teuthology.orchestra.run.smithi072.stdout:Using /usr/lib/python3.6/site-packages 2023-03-20T10:29:12.118 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for urllib3==1.26.15 2023-03-20T10:29:12.118 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: urllib3 1.26.15 2023-03-20T10:29:12.118 INFO:teuthology.orchestra.run.smithi072.stdout:Adding urllib3 1.26.15 to easy-install.pth file 2023-03-20T10:29:12.119 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:12.119 INFO:teuthology.orchestra.run.smithi072.stdout:Using /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib/python3.6/site-packages 2023-03-20T10:29:12.119 INFO:teuthology.orchestra.run.smithi072.stdout:Searching for six==1.11.0 2023-03-20T10:29:13.033 INFO:teuthology.orchestra.run.smithi072.stdout:Best match: six 1.11.0 2023-03-20T10:29:13.033 INFO:teuthology.orchestra.run.smithi072.stdout:Adding six 1.11.0 to easy-install.pth file 2023-03-20T10:29:13.034 INFO:teuthology.orchestra.run.smithi072.stdout: 2023-03-20T10:29:13.034 INFO:teuthology.orchestra.run.smithi072.stdout:Using /usr/lib/python3.6/site-packages 2023-03-20T10:29:13.034 INFO:teuthology.orchestra.run.smithi072.stdout:Finished processing dependencies for bn-tests==0.0.1 2023-03-20T10:29:13.034 INFO:tasks.notification_tests:Running bucket-notifications-tests... 2023-03-20T10:29:15.652 DEBUG:teuthology.orchestra.run.smithi072:bucket notification tests against different endpoints> BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a amqp_test 2023-03-20T10:29:25.671 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification filter on master ... /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/ 2023-03-20T10:29:26.505 INFO:teuthology.orchestra.run.smithi072.stderr: warnings.warn(warning, PythonDeprecationWarning) 2023-03-20T10:29:26.505 INFO:teuthology.orchestra.run.smithi072.stderr:ok 2023-03-20T10:29:38.436 INFO:teuthology.orchestra.run.smithi072.stderr:test pushing amqp s3 notification on master ... ok 2023-03-20T10:29:46.328 INFO:teuthology.orchestra.run.smithi072.stderr:bucket_notification.test_bn.test_ps_s3_creation_triggers_on_master ... ok 2023-03-20T10:29:48.214 INFO:teuthology.orchestra.run.smithi072.stderr:test that uploads object using HTTP POST ... /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version! 2023-03-20T10:29:48.214 INFO:teuthology.orchestra.run.smithi072.stderr: RequestsDependencyWarning) 2023-03-20T10:29:48.214 INFO:teuthology.orchestra.run.smithi072.stderr:/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/lib64/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/ 2023-03-20T10:29:48.214 INFO:teuthology.orchestra.run.smithi072.stderr: warnings.warn(warning, PythonDeprecationWarning) 2023-03-20T10:29:48.214 INFO:teuthology.orchestra.run.smithi072.stderr:ok 2023-03-20T10:29:52.305 INFO:teuthology.orchestra.run.smithi072.stderr:test multipart object upload on master ... ok 2023-03-20T10:30:04.141 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification of metadata on master ... ok 2023-03-20T10:30:16.141 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification of metadata on master ... ok 2023-03-20T10:30:27.772 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification of tags on master ... ok 2023-03-20T10:30:33.514 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification of object versions ... ok 2023-03-20T10:30:41.292 INFO:teuthology.orchestra.run.smithi072.stderr:test s3 notification of deletion markers on master ... ok 2023-03-20T10:32:03.786 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'GET /xibrdv-1?notification=xibrdv-1_notif_4 HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:WNUYf56OrOPMZcBrAjSKMHKFqDM=\r\nDate: Mon, 20 Mar 2023 10:29:16 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx0000069fe136da7645b1d-006418357c-1191-default 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Type: application/xml 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 504 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:16 GMT 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:09.344 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-1?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:yP6QKMoIazBYL7PNtGOQ1/GuCbY=\r\nDate: Mon, 20 Mar 2023 10:29:25 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000c2946806b024bd74-0064183585-1191-default 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:25 GMT 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-1?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:yP6QKMoIazBYL7PNtGOQ1/GuCbY=\r\nDate: Mon, 20 Mar 2023 10:29:25 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000b156d13bb02cb0ca-0064183585-1191-default 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:25 GMT 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:average time for creation + qmqp notification is: 23.090474605560303 milliseconds 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:09.345 INFO:teuthology.orchestra.run.smithi072.stdout:total number of objects: 100 2023-03-20T10:32:09.346 INFO:teuthology.orchestra.run.smithi072.stdout:average time for deletion + amqp notification is: 2.930889129638672 milliseconds 2023-03-20T10:32:09.346 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:09.346 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-2?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:F5LU3vnDEz/9d1VyJsFvuZJ1UAA=\r\nDate: Mon, 20 Mar 2023 10:29:38 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:09.346 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:09.346 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:10.810 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx0000079e70560f489f652-0064183592-1191-default 2023-03-20T10:32:12.150 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:12.150 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:38 GMT 2023-03-20T10:32:12.150 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:12.150 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:12.150 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-3?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:SUFSgkPFFHA6L7Rub052AoS6WMo=\r\nDate: Mon, 20 Mar 2023 10:29:46 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000ee76382cecef0abe-006418359a-1191-default 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:46 GMT 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-4?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:pkC2hpz5RZrzDzyZAwq6VW0w3x4=\r\nDate: Mon, 20 Mar 2023 10:29:46 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:12.151 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000ba8c8616553f128b-006418359a-1191-default 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:46 GMT 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-5?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:whaeBNB9wBP2upThElL0xHrK2ts=\r\nDate: Mon, 20 Mar 2023 10:29:52 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:12.152 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx000008f7a126d70b9badd-00641835a0-1191-default 2023-03-20T10:32:13.825 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:13.825 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:29:52 GMT 2023-03-20T10:32:13.826 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:13.826 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:13.826 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:13.826 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-6?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:u3PeFr1IugSdKiYTvcKWC7/vhwQ=\r\nDate: Mon, 20 Mar 2023 10:30:04 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:13.826 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx0000018ff120bbff7b6c2-00641835ac-1191-default 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:30:04 GMT 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:23.653 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-7?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:75qDMGLMLfMyRFfbD3320uOq1Ek=\r\nDate: Mon, 20 Mar 2023 10:30:16 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000fb90f081e2fbdfc1-00641835b8-1191-default 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:30:16 GMT 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:send: b'DELETE /xibrdv-8?notification HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nAuthorization: AWS YDFTONNSXZETVCOKUMGC:425nb9qB/9D+411MnCWwmbdDk4o=\r\nDate: Mon, 20 Mar 2023 10:30:27 +0000\r\nHost: smithi072.front.sepia.ceph.com:80\r\n\r\n' 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:Zero length chunk ignored 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:reply: 'HTTP/1.1 200 OK\r\n' 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: x-amz-request-id: tx00000eb0a00ae8f41412a-00641835c3-1191-default 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Content-Length: 0 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Date: Mon, 20 Mar 2023 10:30:27 GMT 2023-03-20T10:32:23.654 INFO:teuthology.orchestra.run.smithi072.stdout:header: Connection: Keep-Alive 2023-03-20T10:32:25.138 INFO:teuthology.orchestra.run.smithi072.stdout:wait for 5sec for the messages... 2023-03-20T10:32:25.138 INFO:teuthology.orchestra.run.smithi072.stdout:version ok: MpZwaiEi6exUDKpeqRMUdnVMJzITj9B