2023-01-25T12:08:32.978 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T12:08:32.979 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} --archive /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137407 --name yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137407/orig.config.yaml 2023-01-25T12:08:33.006 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T12:08:33.060 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137407 branch: wip-yuri6-testing-2023-01-24-0746 description: rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} email: yweinstein@radhat.com first_in_suite: false job_id: '7137407' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri6-testing-2023-01-24-0746 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri6-testing-2023-01-24-0746 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri6-testing-2023-01-24-0746 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c targets: smithi090.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:27:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T12:08:33.060 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa; will attempt to use it 2023-01-25T12:08:33.061 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa/tasks 2023-01-25T12:08:33.061 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T12:08:33.063 INFO:teuthology.task.internal:Checking packages... 2023-01-25T12:08:33.093 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '44e4095da12a8269eb6cfe163cbb0d00b1a6412c' 2023-01-25T12:08:33.093 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T12:08:33.095 INFO:teuthology.packaging:ref: None 2023-01-25T12:08:33.096 INFO:teuthology.packaging:tag: None 2023-01-25T12:08:33.096 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-01-24-0746 2023-01-25T12:08:33.097 INFO:teuthology.packaging:sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:08:33.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2023-01-24-0746 2023-01-25T12:08:33.261 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1986.g44e4095d 2023-01-25T12:08:33.262 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T12:08:33.270 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T12:08:33.270 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T12:08:33.292 INFO:teuthology.task.internal:Saving configuration 2023-01-25T12:08:33.303 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T12:08:33.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T12:08:33.341 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 12:02:58.917384', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T12:08:33.341 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T12:08:33.354 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-01-25T12:08:33.354 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T12:08:33.400 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1a36d3f5e0>, signals=[15]) 2023-01-25T12:08:33.401 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T12:08:33.412 INFO:teuthology.task.internal:Opening connections... 2023-01-25T12:08:33.413 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2023-01-25T12:08:33.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:08:33.488 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T12:08:33.500 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2023-01-25T12:08:33.520 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2023-01-25T12:08:33.520 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2023-01-25T12:08:33.582 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T12:08:33.583 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T12:08:33.583 INFO:teuthology.orchestra.run.smithi090.stdout:ID="rhel" 2023-01-25T12:08:33.583 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="fedora" 2023-01-25T12:08:33.583 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8.6" 2023-01-25T12:08:33.584 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8" 2023-01-25T12:08:33.584 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T12:08:33.584 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2023-01-25T12:08:33.584 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T12:08:33.585 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T12:08:33.586 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T12:08:33.586 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T12:08:33.587 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2023-01-25T12:08:33.604 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T12:08:33.617 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T12:08:33.630 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T12:08:33.631 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2023-01-25T12:08:33.648 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T12:08:33.659 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T12:08:33.660 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2023-01-25T12:08:33.718 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T12:08:33.785 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T12:08:33.810 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T12:08:33.811 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T12:08:33.811 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T12:08:33.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:08:33.828 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2023-01-25T12:08:33.889 INFO:teuthology.orchestra.run.smithi090.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T12:08:33.890 INFO:teuthology.task.kernel:Running kernel on smithi090: 4.18.0-372.9.1.el8.x86_64 2023-01-25T12:08:33.890 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2023-01-25T12:08:35.266 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:08:35.267 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-01-25T12:08:37.261 INFO:teuthology.orchestra.run.smithi090.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T12:08:38.763 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 13 MB 00:01 2023-01-25T12:08:41.697 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 534 kB/s | 24 kB 00:00 2023-01-25T12:08:42.366 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T12:08:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:08:42.421 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:08:42.421 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:08:42.534 DEBUG:teuthology.orchestra.run.smithi090:> echo no | sudo yum reinstall kernel || true 2023-01-25T12:08:42.865 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:08:42.865 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-01-25T12:08:43.118 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 12:08:41 PM UTC. 2023-01-25T12:08:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T12:08:43.234 INFO:teuthology.orchestra.run.smithi090.stderr:Error: No packages marked for reinstall. 2023-01-25T12:08:43.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum reinstall -y kernel || true 2023-01-25T12:08:43.609 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:08:43.610 INFO:teuthology.orchestra.run.smithi090.stdout:Unable to read consumer identity 2023-01-25T12:08:43.868 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 12:08:41 PM UTC. 2023-01-25T12:08:43.913 INFO:teuthology.orchestra.run.smithi090.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T12:08:43.939 INFO:teuthology.orchestra.run.smithi090.stderr:Error: No packages marked for reinstall. 2023-01-25T12:08:43.990 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel --last | head -n 1 2023-01-25T12:08:44.069 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T12:08:44.071 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T12:08:44.072 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T12:08:44.072 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T12:08:44.073 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T12:08:44.108 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2023-01-25T12:08:44.129 DEBUG:teuthology.parallel:result is None 2023-01-25T12:08:44.129 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T12:08:44.137 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T12:08:44.138 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T12:08:44.185 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T12:08:44.193 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T12:08:44.201 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T12:08:44.201 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T12:08:44.245 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T12:08:44.253 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T12:08:44.254 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T12:08:44.318 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:08:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:08:44.329 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T12:08:44.342 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T12:08:44.343 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T12:08:44.396 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T12:08:44.406 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T12:08:44.406 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T12:08:44.453 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:08:44.534 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T12:08:44.593 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:08:44.678 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T12:08:44.720 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:08:44.721 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T12:08:44.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2023-01-25T12:08:44.895 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T12:08:45.287 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T12:08:45.296 INFO:teuthology.task.internal:Starting timer... 2023-01-25T12:08:45.296 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T12:08:45.308 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T12:08:45.318 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T12:08:45.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2023-01-25T12:08:45.378 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2023-01-25T12:08:45.380 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T12:08:45.381 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2023-01-25T12:08:45.399 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2023-01-25T12:08:45.400 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive'} 2023-01-25T12:08:45.400 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T12:08:45.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:08:45.468 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T12:08:45.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2023-01-25T12:08:45.537 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T12:08:45.548 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T12:08:45.549 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T12:08:45.573 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T12:08:45.574 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T12:15:26.224 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com')] 2023-01-25T12:15:26.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2023-01-25T12:15:26.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T12:15:26.297 DEBUG:teuthology.orchestra.run.smithi090:> true 2023-01-25T12:15:26.361 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T12:15:26.371 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T12:15:26.372 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T12:15:26.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T12:15:26.431 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T12:15:26.445 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T12:15:26.533 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2023-01-25T12:15:26.545 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2023-01-25T12:15:26.559 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T12:15:26.572 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T12:15:26.646 INFO:teuthology.orchestra.run.smithi090.stderr:bash: ntpq: command not found 2023-01-25T12:15:26.649 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T12:15:26.650 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2023-01-25T12:15:26.650 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:15:26.650 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:15:26.650 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:15:26.650 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T12:15:26.651 INFO:teuthology.run_tasks:Running task install... 2023-01-25T12:15:26.660 DEBUG:teuthology.task.install:project ceph 2023-01-25T12:15:26.660 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T12:15:26.661 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'} 2023-01-25T12:15:26.661 INFO:teuthology.task.install:Using flavor: default 2023-01-25T12:15:26.667 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T12:15:26.668 INFO:teuthology.task.install:extra packages: [] 2023-01-25T12:15:26.668 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2023-01-25T12:15:26.732 INFO:teuthology.orchestra.run.smithi090.stdout:RedHatEnterprise 2023-01-25T12:15:26.733 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T12:15:26.734 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T12:15:26.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2023-01-25T12:15:26.813 INFO:teuthology.orchestra.run.smithi090.stdout:RedHatEnterprise 2023-01-25T12:15:26.814 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T12:15:26.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:15:26.967 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T12:15:26.968 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T12:15:27.083 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T12:15:27.084 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:15:27.084 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T12:15:27.111 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T12:15:27.112 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T12:15:27.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T12:15:27.256 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T12:15:27.322 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2023-01-25T12:15:27.324 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2023-01-25T12:15:27.693 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:15:27.971 INFO:teuthology.orchestra.run.smithi090.stdout:44 files removed 2023-01-25T12:15:28.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw 2023-01-25T12:15:28.346 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:15:28.991 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 272 kB/s | 79 kB 00:00 2023-01-25T12:15:29.267 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 38 kB/s | 9.3 kB 00:00 2023-01-25T12:15:29.543 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2023-01-25T12:15:29.719 INFO:teuthology.orchestra.run.smithi090.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-01-25T12:15:31.203 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 13 MB 00:01 2023-01-25T12:15:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 330 kB/s | 24 kB 00:00 2023-01-25T12:15:35.199 INFO:teuthology.orchestra.run.smithi090.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 47 MB 00:00 2023-01-25T12:15:43.201 INFO:teuthology.orchestra.run.smithi090.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 53 MB 00:00 2023-01-25T12:15:54.723 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:15:54.725 INFO:teuthology.orchestra.run.smithi090.stdout:==================================================================================================== 2023-01-25T12:15:54.725 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:15:54.726 INFO:teuthology.orchestra.run.smithi090.stdout:==================================================================================================== 2023-01-25T12:15:54.726 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:15:54.726 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T12:15:54.726 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2023-01-25T12:15:54.727 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:15:54.727 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T12:15:54.727 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:15:54.727 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T12:15:54.728 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T12:15:54.728 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T12:15:54.728 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T12:15:54.728 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T12:15:54.728 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T12:15:54.729 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T12:15:54.729 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T12:15:54.729 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T12:15:54.729 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T12:15:54.729 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T12:15:54.730 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T12:15:54.730 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T12:15:54.730 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T12:15:54.730 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T12:15:54.730 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T12:15:54.731 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T12:15:54.731 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T12:15:54.731 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T12:15:54.731 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T12:15:54.731 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:15:54.732 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:15:54.732 INFO:teuthology.orchestra.run.smithi090.stdout:==================================================================================================== 2023-01-25T12:15:54.732 INFO:teuthology.orchestra.run.smithi090.stdout:Install 21 Packages 2023-01-25T12:15:54.732 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 2 Packages 2023-01-25T12:15:54.732 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:15:54.733 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 58 M 2023-01-25T12:15:54.733 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:15:55.819 INFO:teuthology.orchestra.run.smithi090.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 4.5 MB/s | 4.8 MB 00:01 2023-01-25T12:15:55.894 INFO:teuthology.orchestra.run.smithi090.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 325 kB/s | 24 kB 00:00 2023-01-25T12:15:56.091 INFO:teuthology.orchestra.run.smithi090.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 3.8 MB/s | 767 kB 00:00 2023-01-25T12:15:56.220 INFO:teuthology.orchestra.run.smithi090.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 3.6 MB/s | 474 kB 00:00 2023-01-25T12:15:56.871 INFO:teuthology.orchestra.run.smithi090.stdout:(5/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8. 6.1 MB/s | 13 MB 00:02 2023-01-25T12:15:56.954 INFO:teuthology.orchestra.run.smithi090.stdout:(6/23): python3-ceph-argparse-18.0.0-1986.g44e4 536 kB/s | 45 kB 00:00 2023-01-25T12:15:57.063 INFO:teuthology.orchestra.run.smithi090.stdout:(7/23): python3-ceph-common-18.0.0-1986.g44e409 1.0 MB/s | 115 kB 00:00 2023-01-25T12:15:57.364 INFO:teuthology.orchestra.run.smithi090.stdout:(8/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 5.7 MB/s | 6.5 MB 00:01 2023-01-25T12:15:57.390 INFO:teuthology.orchestra.run.smithi090.stdout:(9/23): python3-cephfs-18.0.0-1986.g44e4095d.el 583 kB/s | 190 kB 00:00 2023-01-25T12:15:57.457 INFO:teuthology.orchestra.run.smithi090.stdout:(10/23): python3-rados-18.0.0-1986.g44e4095d.el 4.0 MB/s | 372 kB 00:00 2023-01-25T12:15:57.516 INFO:teuthology.orchestra.run.smithi090.stdout:(11/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 2.8 MB/s | 364 kB 00:00 2023-01-25T12:15:57.549 INFO:teuthology.orchestra.run.smithi090.stdout:(12/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.1 MB/s | 106 kB 00:00 2023-01-25T12:15:57.892 INFO:teuthology.orchestra.run.smithi090.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 172 kB/s | 59 kB 00:00 2023-01-25T12:15:58.167 INFO:teuthology.orchestra.run.smithi090.stdout:(14/23): ceph-common-18.0.0-1986.g44e4095d.el8. 6.2 MB/s | 21 MB 00:03 2023-01-25T12:15:58.192 INFO:teuthology.orchestra.run.smithi090.stdout:(15/23): libunwind-1.3.1-3.el8.x86_64.rpm 250 kB/s | 75 kB 00:00 2023-01-25T12:15:58.251 INFO:teuthology.orchestra.run.smithi090.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.4 MB/s | 259 kB 00:00 2023-01-25T12:15:58.293 INFO:teuthology.orchestra.run.smithi090.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-01-25T12:15:58.335 INFO:teuthology.orchestra.run.smithi090.stdout:(18/23): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-01-25T12:15:58.376 INFO:teuthology.orchestra.run.smithi090.stdout:(19/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-01-25T12:15:58.444 INFO:teuthology.orchestra.run.smithi090.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 330 kB/s | 306 kB 00:00 2023-01-25T12:15:58.910 INFO:teuthology.orchestra.run.smithi090.stdout:(21/23): librados2-18.0.0-1986.g44e4095d.el8.x8 6.8 MB/s | 3.6 MB 00:00 2023-01-25T12:15:59.061 INFO:teuthology.orchestra.run.smithi090.stdout:(22/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 5.8 MB/s | 3.6 MB 00:00 2023-01-25T12:15:59.403 INFO:teuthology.orchestra.run.smithi090.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.4 MB/s | 1.7 MB 00:01 2023-01-25T12:15:59.405 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:15:59.405 INFO:teuthology.orchestra.run.smithi090.stdout:Total 12 MB/s | 58 MB 00:04 2023-01-25T12:15:59.453 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:15:59.499 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:15:59.499 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:15:59.846 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:15:59.850 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:16:01.059 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:16:01.354 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T12:16:01.549 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T12:16:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T12:16:02.207 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T12:16:02.312 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T12:16:02.538 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:16:02.830 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T12:16:03.032 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:16:03.483 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T12:16:03.723 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:16:03.788 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:16:03.999 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T12:16:04.207 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:16:04.259 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T12:16:04.409 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T12:16:04.977 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:16:05.248 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:16:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:16:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T12:16:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T12:16:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T12:16:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T12:16:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T12:16:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T12:16:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 18/25 2023-01-25T12:16:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T12:16:11.149 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:16:11.399 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:16:11.958 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T12:16:12.276 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:16:12.355 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T12:16:12.557 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:16:31.942 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:16:31.942 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2023-01-25T12:16:31.942 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2023-01-25T12:16:31.943 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /dev 2023-01-25T12:16:31.943 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /run 2023-01-25T12:16:31.943 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2023-01-25T12:16:31.943 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2023-01-25T12:16:31.943 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2023-01-25T12:16:31.944 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2023-01-25T12:16:31.944 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /dev 2023-01-25T12:16:31.944 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:33.317 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:16:33.338 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T12:16:33.338 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:16:33.591 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T12:16:33.592 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:16:34.064 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:16:34.064 INFO:teuthology.orchestra.run.smithi090.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T12:16:34.065 INFO:teuthology.orchestra.run.smithi090.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T12:16:34.065 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:34.215 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T12:16:34.216 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T12:16:34.216 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T12:16:34.216 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T12:16:34.217 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T12:16:34.217 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T12:16:34.217 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T12:16:34.217 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T12:16:34.217 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T12:16:34.218 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T12:16:34.218 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T12:16:34.218 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T12:16:34.218 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T12:16:34.218 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T12:16:34.219 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T12:16:34.219 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T12:16:34.219 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T12:16:34.219 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 18/25 2023-01-25T12:16:34.219 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-25T12:16:34.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2023-01-25T12:16:34.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 21/25 2023-01-25T12:16:34.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T12:16:34.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T12:16:34.220 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T12:16:35.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T12:16:35.617 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:53 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:16:37.972 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:16:38.173 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:38.174 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2023-01-25T12:16:38.174 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.174 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.175 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:16:38.175 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.176 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.176 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.176 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.177 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T12:16:38.177 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.177 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T12:16:38.178 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T12:16:38.178 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.178 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T12:16:38.178 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.179 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T12:16:38.180 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T12:16:38.180 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.180 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T12:16:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.181 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:38.182 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T12:16:38.182 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:38.182 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:16:38.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-test 2023-01-25T12:16:38.712 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:16:39.506 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:16:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:16:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2023-01-25T12:16:40.933 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:16:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2023-01-25T12:16:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:16:40.934 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T12:16:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:16:40.934 INFO:teuthology.orchestra.run.smithi090.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T12:16:40.935 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T12:16:40.935 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T12:16:40.935 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T12:16:40.935 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T12:16:40.935 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:16:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2023-01-25T12:16:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2023-01-25T12:16:40.936 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 46 M 2023-01-25T12:16:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 204 M 2023-01-25T12:16:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:16:40.974 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.8 MB/s | 70 kB 00:00 2023-01-25T12:16:41.049 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-01-25T12:16:41.091 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-01-25T12:16:41.133 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-01-25T12:16:41.167 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 595 kB/s | 137 kB 00:00 2023-01-25T12:16:43.433 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 18 MB/s | 45 MB 00:02 2023-01-25T12:16:43.436 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:16:43.436 INFO:teuthology.orchestra.run.smithi090.stdout:Total 18 MB/s | 46 MB 00:02 2023-01-25T12:16:43.474 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:16:43.496 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:16:43.496 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:16:43.745 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:16:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:16:44.204 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:16:44.364 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T12:16:44.438 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T12:16:44.657 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T12:16:44.848 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T12:16:45.017 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T12:16:45.247 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:16:49.587 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T12:16:49.811 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:16:50.034 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T12:16:50.034 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T12:16:50.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T12:16:50.035 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T12:16:50.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T12:16:50.036 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T12:16:51.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T12:16:51.112 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:09 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:16:53.609 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:16:53.790 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:53.790 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:16:53.790 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:53.791 INFO:teuthology.orchestra.run.smithi090.stdout: jq-1.6-3.el8.x86_64 2023-01-25T12:16:53.791 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:16:53.791 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T12:16:53.791 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T12:16:53.791 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T12:16:53.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:53.792 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:16:53.924 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph 2023-01-25T12:16:54.258 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:16:55.046 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:16:56.461 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:16:56.464 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================= 2023-01-25T12:16:56.464 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================= 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T12:16:56.465 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T12:16:56.466 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T12:16:56.466 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T12:16:56.466 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T12:16:56.466 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T12:16:56.466 INFO:teuthology.orchestra.run.smithi090.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T12:16:56.467 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T12:16:56.467 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T12:16:56.467 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T12:16:56.467 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T12:16:56.467 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T12:16:56.468 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T12:16:56.468 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T12:16:56.468 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T12:16:56.468 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T12:16:56.468 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T12:16:56.469 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T12:16:56.469 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T12:16:56.469 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T12:16:56.469 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T12:16:56.470 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T12:16:56.470 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T12:16:56.470 INFO:teuthology.orchestra.run.smithi090.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T12:16:56.470 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T12:16:56.470 INFO:teuthology.orchestra.run.smithi090.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T12:16:56.471 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:56.472 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:16:56.472 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================= 2023-01-25T12:16:56.472 INFO:teuthology.orchestra.run.smithi090.stdout:Install 32 Packages 2023-01-25T12:16:56.472 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:16:56.473 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 28 M 2023-01-25T12:16:56.473 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 107 M 2023-01-25T12:16:56.473 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:16:56.619 INFO:teuthology.orchestra.run.smithi090.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 43 kB/s | 6.4 kB 00:00 2023-01-25T12:16:56.836 INFO:teuthology.orchestra.run.smithi090.stdout:(2/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 6.2 MB/s | 2.3 MB 00:00 2023-01-25T12:16:56.878 INFO:teuthology.orchestra.run.smithi090.stdout:(3/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 3.8 MB/s | 1.5 MB 00:00 2023-01-25T12:16:56.978 INFO:teuthology.orchestra.run.smithi090.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 2.3 MB/s | 236 kB 00:00 2023-01-25T12:16:57.112 INFO:teuthology.orchestra.run.smithi090.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 327 kB/s | 44 kB 00:00 2023-01-25T12:16:57.222 INFO:teuthology.orchestra.run.smithi090.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T12:16:57.272 INFO:teuthology.orchestra.run.smithi090.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 3.4 MB/s | 173 kB 00:00 2023-01-25T12:16:57.364 INFO:teuthology.orchestra.run.smithi090.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 4.1 MB/s | 384 kB 00:00 2023-01-25T12:16:57.423 INFO:teuthology.orchestra.run.smithi090.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-01-25T12:16:57.473 INFO:teuthology.orchestra.run.smithi090.stdout:(10/32): python3-jaraco-functools-2.0-4.el8.noa 351 kB/s | 18 kB 00:00 2023-01-25T12:16:57.523 INFO:teuthology.orchestra.run.smithi090.stdout:(11/32): python3-logutils-0.3.5-11.el8.noarch.r 974 kB/s | 49 kB 00:00 2023-01-25T12:16:57.724 INFO:teuthology.orchestra.run.smithi090.stdout:(12/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 19 MB/s | 17 MB 00:00 2023-01-25T12:16:57.749 INFO:teuthology.orchestra.run.smithi090.stdout:(13/32): python3-more-itertools-7.2.0-3.el8.noa 260 kB/s | 59 kB 00:00 2023-01-25T12:16:57.816 INFO:teuthology.orchestra.run.smithi090.stdout:(14/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86 3.2 MB/s | 3.9 MB 00:01 2023-01-25T12:16:57.872 INFO:teuthology.orchestra.run.smithi090.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 128 kB/s | 16 kB 00:00 2023-01-25T12:16:57.934 INFO:teuthology.orchestra.run.smithi090.stdout:(16/32): python3-singledispatch-3.4.0.3-18.el8. 387 kB/s | 24 kB 00:00 2023-01-25T12:16:57.959 INFO:teuthology.orchestra.run.smithi090.stdout:(17/32): python3-simplegeneric-0.8.1-17.el8.noa 131 kB/s | 19 kB 00:00 2023-01-25T12:16:57.984 INFO:teuthology.orchestra.run.smithi090.stdout:(18/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-01-25T12:16:58.009 INFO:teuthology.orchestra.run.smithi090.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 387 kB/s | 29 kB 00:00 2023-01-25T12:16:58.043 INFO:teuthology.orchestra.run.smithi090.stdout:(20/32): python3-trustme-0.6.0-4.el8.noarch.rpm 327 kB/s | 27 kB 00:00 2023-01-25T12:16:58.069 INFO:teuthology.orchestra.run.smithi090.stdout:(21/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.8 MB/s | 241 kB 00:00 2023-01-25T12:16:58.094 INFO:teuthology.orchestra.run.smithi090.stdout:(22/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.9 MB/s | 251 kB 00:00 2023-01-25T12:16:58.119 INFO:teuthology.orchestra.run.smithi090.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 456 kB/s | 23 kB 00:00 2023-01-25T12:16:58.144 INFO:teuthology.orchestra.run.smithi090.stdout:(24/32): python3-webtest-2.0.33-1.el8.noarch.rp 847 kB/s | 85 kB 00:00 2023-01-25T12:16:58.203 INFO:teuthology.orchestra.run.smithi090.stdout:(25/32): python3-pytz-2017.2-9.el8.noarch.rpm 496 kB/s | 54 kB 00:00 2023-01-25T12:16:58.229 INFO:teuthology.orchestra.run.smithi090.stdout:(26/32): python3-markupsafe-0.23-19.el8.x86_64. 359 kB/s | 39 kB 00:00 2023-01-25T12:16:58.254 INFO:teuthology.orchestra.run.smithi090.stdout:(27/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.6 MB/s | 157 kB 00:00 2023-01-25T12:16:58.288 INFO:teuthology.orchestra.run.smithi090.stdout:(28/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 1.7 MB/s | 103 kB 00:00 2023-01-25T12:16:58.355 INFO:teuthology.orchestra.run.smithi090.stdout:(29/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.0 MB/s | 457 kB 00:00 2023-01-25T12:16:58.381 INFO:teuthology.orchestra.run.smithi090.stdout:(30/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-01-25T12:16:58.415 INFO:teuthology.orchestra.run.smithi090.stdout:(31/32): python3-pycparser-2.14-14.el8.noarch.r 859 kB/s | 109 kB 00:00 2023-01-25T12:16:58.448 INFO:teuthology.orchestra.run.smithi090.stdout:(32/32): python3-cryptography-3.2.1-5.el8.x86_6 5.9 MB/s | 559 kB 00:00 2023-01-25T12:16:58.450 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:16:58.450 INFO:teuthology.orchestra.run.smithi090.stdout:Total 14 MB/s | 28 MB 00:01 2023-01-25T12:16:58.543 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:16:58.579 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:16:58.579 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:16:58.780 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:16:58.784 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:16:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:17:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T12:17:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T12:17:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2023-01-25T12:17:00.951 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 4/32 2023-01-25T12:17:01.208 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 5/32 2023-01-25T12:17:01.379 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2023-01-25T12:17:01.622 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2023-01-25T12:17:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2023-01-25T12:17:02.142 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 9/32 2023-01-25T12:17:02.368 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2023-01-25T12:17:02.596 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 11/32 2023-01-25T12:17:02.754 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 12/32 2023-01-25T12:17:02.978 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T12:17:03.153 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T12:17:03.325 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T12:17:03.516 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T12:17:03.674 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T12:17:03.827 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T12:17:04.125 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T12:17:04.308 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T12:17:04.519 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T12:17:04.792 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T12:17:05.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T12:17:05.257 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T12:17:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T12:17:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T12:17:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T12:17:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:17:07.919 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T12:17:08.157 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:17:08.540 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T12:17:08.721 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:17:08.935 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T12:17:09.146 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:17:09.165 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T12:17:09.269 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:17:09.607 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T12:17:09.610 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T12:17:09.610 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T12:17:09.611 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T12:17:09.611 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T12:17:09.611 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T12:17:09.611 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T12:17:09.611 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T12:17:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T12:17:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T12:17:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T12:17:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T12:17:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T12:17:09.613 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T12:17:09.613 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T12:17:09.613 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T12:17:09.613 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T12:17:09.613 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T12:17:09.614 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T12:17:09.614 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T12:17:09.614 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T12:17:09.614 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T12:17:09.614 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T12:17:09.615 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T12:17:09.615 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T12:17:09.615 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 25/32 2023-01-25T12:17:09.615 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2023-01-25T12:17:09.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/32 2023-01-25T12:17:09.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2023-01-25T12:17:09.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 29/32 2023-01-25T12:17:09.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 30/32 2023-01-25T12:17:09.616 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 31/32 2023-01-25T12:17:11.064 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 32/32 2023-01-25T12:17:11.065 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:29 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:13.525 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:17:13.711 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:13.712 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:17:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:13.712 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:17:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:13.713 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T12:17:13.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T12:17:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T12:17:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T12:17:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T12:17:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T12:17:13.715 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T12:17:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T12:17:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T12:17:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T12:17:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T12:17:13.716 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T12:17:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T12:17:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T12:17:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T12:17:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T12:17:13.717 INFO:teuthology.orchestra.run.smithi090.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T12:17:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T12:17:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T12:17:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T12:17:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T12:17:13.718 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:13.719 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:13.874 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-base 2023-01-25T12:17:14.219 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:15.025 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:33 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:16.310 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:17:16.420 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:17:16.422 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:16.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install cephadm 2023-01-25T12:17:16.884 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:17.750 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:35 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:19.138 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:19.139 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:19.139 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:17:19.140 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:19.140 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:17:19.140 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T12:17:19.140 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:19.140 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:17:19.141 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:19.141 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:17:19.141 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:19.141 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 82 k 2023-01-25T12:17:19.142 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 374 k 2023-01-25T12:17:19.142 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:17:19.454 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 262 kB/s | 82 kB 00:00 2023-01-25T12:17:19.455 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:19.455 INFO:teuthology.orchestra.run.smithi090.stdout:Total 261 kB/s | 82 kB 00:00 2023-01-25T12:17:19.455 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:17:19.466 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:17:19.467 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:17:19.468 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:17:19.470 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:17:19.512 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:17:21.102 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:17:21.252 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:17:21.419 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:17:22.471 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:17:22.472 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:25.143 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:17:25.324 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:25.325 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:17:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:17:25.325 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:25.325 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:25.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T12:17:25.836 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:26.634 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:28.008 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:28.009 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:28.009 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2023-01-25T12:17:28.009 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:28.009 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:17:28.010 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T12:17:28.010 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:28.010 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:17:28.010 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:17:28.011 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:17:28.011 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:28.011 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 165 k 2023-01-25T12:17:28.011 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 503 k 2023-01-25T12:17:28.011 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:17:28.214 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 808 kB/s | 165 kB 00:00 2023-01-25T12:17:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:Total 803 kB/s | 165 kB 00:00 2023-01-25T12:17:28.216 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:17:28.226 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:17:28.227 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:17:28.278 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:17:28.281 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:17:28.396 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:17:28.612 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:17:28.972 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:17:29.965 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T12:17:29.965 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:47 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:32.288 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:17:32.467 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:32.467 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:17:32.468 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:17:32.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:32.469 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:32.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr 2023-01-25T12:17:32.954 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:33.747 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:51 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:35.031 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:17:35.137 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:35.138 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:17:35.138 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:35.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T12:17:35.750 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:36.525 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:54 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:37.902 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:37.903 INFO:teuthology.orchestra.run.smithi090.stdout:=================================================================================================== 2023-01-25T12:17:37.903 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:17:37.903 INFO:teuthology.orchestra.run.smithi090.stdout:=================================================================================================== 2023-01-25T12:17:37.904 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:17:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T12:17:37.904 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:17:37.904 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T12:17:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T12:17:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T12:17:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T12:17:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T12:17:37.905 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:37.906 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:17:37.906 INFO:teuthology.orchestra.run.smithi090.stdout:=================================================================================================== 2023-01-25T12:17:37.906 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2023-01-25T12:17:37.906 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:37.907 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.0 M 2023-01-25T12:17:37.907 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 8.2 M 2023-01-25T12:17:37.907 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:17:38.091 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 124 kB/s | 23 kB 00:00 2023-01-25T12:17:38.108 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 72 kB/s | 15 kB 00:00 2023-01-25T12:17:38.209 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 288 kB/s | 34 kB 00:00 2023-01-25T12:17:38.259 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-01-25T12:17:38.284 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.1 MB/s | 196 kB 00:00 2023-01-25T12:17:38.893 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 1.7 MB/s | 1.7 MB 00:00 2023-01-25T12:17:38.894 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:38.895 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.0 MB/s | 2.0 MB 00:00 2023-01-25T12:17:38.907 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:17:38.916 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:17:38.916 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:17:38.975 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:17:38.978 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:17:39.222 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:17:39.420 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T12:17:39.583 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T12:17:39.789 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T12:17:39.934 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T12:17:40.293 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T12:17:40.522 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:17:40.753 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T12:17:40.754 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T12:17:40.754 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T12:17:40.754 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T12:17:40.754 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T12:17:40.754 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T12:17:42.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T12:17:42.619 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:00 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:45.469 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:17:45.651 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:17:45.651 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:17:45.652 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:17:45.652 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:17:45.652 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T12:17:45.653 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T12:17:45.654 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T12:17:45.654 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:17:45.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T12:17:46.146 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:17:46.959 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:17:48.337 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:17:48.339 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================================= 2023-01-25T12:17:48.339 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:17:48.339 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================================= 2023-01-25T12:17:48.339 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:17:48.339 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T12:17:48.340 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:17:48.340 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T12:17:48.340 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T12:17:48.340 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T12:17:48.341 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T12:17:48.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T12:17:48.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T12:17:48.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T12:17:48.341 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:48.342 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:17:48.342 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================================= 2023-01-25T12:17:48.342 INFO:teuthology.orchestra.run.smithi090.stdout:Install 8 Packages 2023-01-25T12:17:48.342 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:17:48.343 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 35 M 2023-01-25T12:17:48.343 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 210 M 2023-01-25T12:17:48.343 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:17:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:(1/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 23 MB/s | 3.7 MB 00:00 2023-01-25T12:17:48.545 INFO:teuthology.orchestra.run.smithi090.stdout:(2/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-25T12:17:48.724 INFO:teuthology.orchestra.run.smithi090.stdout:(3/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 27 MB/s | 4.9 MB 00:00 2023-01-25T12:17:48.895 INFO:teuthology.orchestra.run.smithi090.stdout:(4/8): python3-scipy-1.0.0-21.module+el8.5.0+10 25 MB/s | 14 MB 00:00 2023-01-25T12:17:48.954 INFO:teuthology.orchestra.run.smithi090.stdout:(5/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 11 MB/s | 644 kB 00:00 2023-01-25T12:17:48.996 INFO:teuthology.orchestra.run.smithi090.stdout:(6/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 4.0 MB/s | 171 kB 00:00 2023-01-25T12:17:49.071 INFO:teuthology.orchestra.run.smithi090.stdout:(7/8): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-01-25T12:17:49.605 INFO:teuthology.orchestra.run.smithi090.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-198 5.9 MB/s | 7.4 MB 00:01 2023-01-25T12:17:49.606 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:17:49.606 INFO:teuthology.orchestra.run.smithi090.stdout:Total 28 MB/s | 35 MB 00:01 2023-01-25T12:17:49.841 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:17:49.859 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:17:49.859 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:17:50.151 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:17:50.154 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:17:50.699 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:17:50.852 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T12:17:50.996 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T12:17:51.161 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T12:17:51.763 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T12:17:51.994 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T12:17:52.590 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T12:17:52.760 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T12:17:53.450 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T12:17:53.747 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T12:17:53.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:17:55.715 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T12:17:56.960 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T12:17:57.144 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:17:57.406 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T12:17:57.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T12:17:57.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 2/8 2023-01-25T12:17:57.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 3/8 2023-01-25T12:17:57.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T12:17:57.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 5/8 2023-01-25T12:17:57.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 6/8 2023-01-25T12:17:57.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 7/8 2023-01-25T12:17:58.465 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 8/8 2023-01-25T12:17:58.465 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:16 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:00.931 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:01.113 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:01.113 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:01.113 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:18:01.114 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T12:18:01.114 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T12:18:01.114 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T12:18:01.114 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T12:18:01.114 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T12:18:01.115 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T12:18:01.115 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T12:18:01.115 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:01.115 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:01.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-rook 2023-01-25T12:18:01.598 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:02.380 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================== 2023-01-25T12:18:03.766 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:18:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================== 2023-01-25T12:18:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:03.766 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T12:18:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:18:03.767 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T12:18:03.767 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T12:18:03.767 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T12:18:03.767 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T12:18:03.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T12:18:03.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T12:18:03.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T12:18:03.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T12:18:03.768 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T12:18:03.769 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T12:18:03.769 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T12:18:03.769 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T12:18:03.769 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:03.769 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:03.770 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================================== 2023-01-25T12:18:03.770 INFO:teuthology.orchestra.run.smithi090.stdout:Install 13 Packages 2023-01-25T12:18:03.770 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:03.771 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 1.9 M 2023-01-25T12:18:03.771 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 25 M 2023-01-25T12:18:03.771 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:03.996 INFO:teuthology.orchestra.run.smithi090.stdout:(1/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 208 kB/s | 47 kB 00:00 2023-01-25T12:18:04.222 INFO:teuthology.orchestra.run.smithi090.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 36 kB/s | 16 kB 00:00 2023-01-25T12:18:04.239 INFO:teuthology.orchestra.run.smithi090.stdout:(3/13): python3-cachetools-3.1.1-4.el8.noarch.r 70 kB/s | 33 kB 00:00 2023-01-25T12:18:04.314 INFO:teuthology.orchestra.run.smithi090.stdout:(4/13): python3-rsa-4.8-1.el8.noarch.rpm 818 kB/s | 61 kB 00:00 2023-01-25T12:18:04.364 INFO:teuthology.orchestra.run.smithi090.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T12:18:04.431 INFO:teuthology.orchestra.run.smithi090.stdout:(6/13): python3-kubernetes-11.0.0-6.el8.noarch. 5.6 MB/s | 1.2 MB 00:00 2023-01-25T12:18:04.457 INFO:teuthology.orchestra.run.smithi090.stdout:(7/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.2 MB/s | 110 kB 00:00 2023-01-25T12:18:04.499 INFO:teuthology.orchestra.run.smithi090.stdout:(8/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 396 kB/s | 27 kB 00:00 2023-01-25T12:18:04.524 INFO:teuthology.orchestra.run.smithi090.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.0 MB/s | 126 kB 00:00 2023-01-25T12:18:04.549 INFO:teuthology.orchestra.run.smithi090.stdout:(10/13): python3-jsonpointer-1.10-11.el8.noarch 397 kB/s | 20 kB 00:00 2023-01-25T12:18:04.575 INFO:teuthology.orchestra.run.smithi090.stdout:(11/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 3.0 MB/s | 155 kB 00:00 2023-01-25T12:18:04.600 INFO:teuthology.orchestra.run.smithi090.stdout:(12/13): python3-requests-oauthlib-1.0.0-1.el8. 851 kB/s | 43 kB 00:00 2023-01-25T12:18:04.775 INFO:teuthology.orchestra.run.smithi090.stdout:(13/13): python3-google-auth-1.1.1-10.el8.noarc 104 kB/s | 81 kB 00:00 2023-01-25T12:18:04.776 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:04.776 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.9 MB/s | 1.9 MB 00:01 2023-01-25T12:18:04.843 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:04.859 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:04.860 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:04.930 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:04.934 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T12:18:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T12:18:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T12:18:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2023-01-25T12:18:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2023-01-25T12:18:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2023-01-25T12:18:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2023-01-25T12:18:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T12:18:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T12:18:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T12:18:08.598 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T12:18:08.801 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T12:18:08.969 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:18:09.207 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T12:18:09.207 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T12:18:09.208 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T12:18:09.208 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T12:18:09.208 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T12:18:09.208 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T12:18:09.209 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T12:18:09.209 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T12:18:09.209 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 8/13 2023-01-25T12:18:09.209 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 9/13 2023-01-25T12:18:09.209 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 10/13 2023-01-25T12:18:09.210 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T12:18:09.210 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2023-01-25T12:18:10.355 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2023-01-25T12:18:10.355 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:28 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:12.897 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:18:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T12:18:13.078 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T12:18:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T12:18:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T12:18:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T12:18:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T12:18:13.079 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T12:18:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T12:18:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T12:18:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T12:18:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T12:18:13.080 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T12:18:13.081 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:13.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T12:18:13.563 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:14.470 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:32 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:15.861 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:15.862 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================================== 2023-01-25T12:18:15.862 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:18:15.862 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================================== 2023-01-25T12:18:15.862 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:15.862 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T12:18:15.863 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:18:15.863 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T12:18:15.863 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T12:18:15.863 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T12:18:15.863 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T12:18:15.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:15.864 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:15.864 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================================== 2023-01-25T12:18:15.864 INFO:teuthology.orchestra.run.smithi090.stdout:Install 5 Packages 2023-01-25T12:18:15.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:15.865 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 6.0 M 2023-01-25T12:18:15.865 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2023-01-25T12:18:15.865 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:16.004 INFO:teuthology.orchestra.run.smithi090.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 431 kB/s | 60 kB 00:00 2023-01-25T12:18:16.071 INFO:teuthology.orchestra.run.smithi090.stdout:(2/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 7.9 MB/s | 538 kB 00:00 2023-01-25T12:18:16.096 INFO:teuthology.orchestra.run.smithi090.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.0 MB/s | 476 kB 00:00 2023-01-25T12:18:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:(4/5): python3-babel-2.5.1-7.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-01-25T12:18:16.397 INFO:teuthology.orchestra.run.smithi090.stdout:(5/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 225 kB/s | 120 kB 00:00 2023-01-25T12:18:16.398 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:16.398 INFO:teuthology.orchestra.run.smithi090.stdout:Total 11 MB/s | 6.0 MB 00:00 2023-01-25T12:18:16.451 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:16.463 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:16.463 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:16.540 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:16.544 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:17.522 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:17.888 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T12:18:18.088 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T12:18:18.456 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T12:18:18.755 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T12:18:18.918 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:18:19.143 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T12:18:19.144 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T12:18:19.144 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T12:18:19.144 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T12:18:19.144 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T12:18:20.166 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T12:18:20.166 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:22.530 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:22.712 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:22.713 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:22.713 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:18:22.713 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T12:18:22.713 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T12:18:22.713 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T12:18:22.714 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T12:18:22.714 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:22.714 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:22.820 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-fuse 2023-01-25T12:18:23.173 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:24.076 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:25.465 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:25.466 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:25.466 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:18:25.466 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:25.467 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse 2023-01-25T12:18:25.467 INFO:teuthology.orchestra.run.smithi090.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T12:18:25.467 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2023-01-25T12:18:25.467 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T12:18:25.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:25.468 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2023-01-25T12:18:25.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:25.469 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 928 k 2023-01-25T12:18:25.469 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 2.9 M 2023-01-25T12:18:25.469 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:25.538 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-01-25T12:18:25.754 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 2.9 MB/s | 845 kB 00:00 2023-01-25T12:18:25.755 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:25.755 INFO:teuthology.orchestra.run.smithi090.stdout:Total 3.1 MB/s | 928 kB 00:00 2023-01-25T12:18:25.766 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:25.787 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:25.788 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:25.861 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:25.864 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:26.029 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:26.274 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T12:18:26.433 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:18:26.767 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T12:18:26.767 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T12:18:27.814 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T12:18:27.814 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:45 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:30.225 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:30.403 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:30.403 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:30.403 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T12:18:30.404 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:30.404 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:30.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-volume 2023-01-25T12:18:30.893 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:31.815 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:49 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:33.188 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:33.189 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2023-01-25T12:18:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:33.190 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T12:18:33.190 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:33.190 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:33.191 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:33.191 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:18:33.191 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:33.192 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 263 k 2023-01-25T12:18:33.192 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 1.3 M 2023-01-25T12:18:33.192 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:33.439 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 1.0 MB/s | 263 kB 00:00 2023-01-25T12:18:33.439 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:33.440 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-25T12:18:33.440 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:33.450 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:33.450 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:33.466 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:33.470 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:33.631 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:33.893 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:18:34.252 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:18:35.304 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T12:18:35.305 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:53 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:37.884 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:38.064 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:38.064 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:38.065 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T12:18:38.065 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:38.065 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:38.206 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados-devel 2023-01-25T12:18:38.562 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:39.349 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:02:57 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:40.729 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:40.730 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2023-01-25T12:18:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:40.730 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:40.730 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T12:18:40.731 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:40.731 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:18:40.731 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:40.732 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 139 k 2023-01-25T12:18:40.732 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 512 k 2023-01-25T12:18:40.732 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 692 kB/s | 139 kB 00:00 2023-01-25T12:18:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:Total 688 kB/s | 139 kB 00:00 2023-01-25T12:18:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:40.993 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:40.997 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:41.114 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:41.290 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:41.558 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:42.821 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:00 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:45.604 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:45.781 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:45.781 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:45.781 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:18:45.782 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:45.782 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:45.912 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs2 2023-01-25T12:18:46.265 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:47.088 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:05 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:48.348 INFO:teuthology.orchestra.run.smithi090.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:18:48.461 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:48.462 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:18:48.462 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:48.574 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs-devel 2023-01-25T12:18:48.928 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:49.798 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:07 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:51.208 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:51.209 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2023-01-25T12:18:51.209 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:18:51.210 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T12:18:51.210 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:18:51.210 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:18:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:18:51.211 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 k 2023-01-25T12:18:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 137 k 2023-01-25T12:18:51.211 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:18:51.369 INFO:teuthology.orchestra.run.smithi090.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 192 kB/s | 30 kB 00:00 2023-01-25T12:18:51.369 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:18:51.369 INFO:teuthology.orchestra.run.smithi090.stdout:Total 191 kB/s | 30 kB 00:00 2023-01-25T12:18:51.370 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:18:51.373 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:18:51.373 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:18:51.383 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:18:51.388 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:18:51.450 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:18:51.639 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:51.854 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:52.847 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:18:52.848 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:10 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:55.616 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:18:55.795 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:55.796 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:18:55.796 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:18:55.796 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:18:55.797 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:55.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados2 2023-01-25T12:18:56.280 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:57.072 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:15 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:18:58.337 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:18:58.446 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:18:58.447 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:18:58.447 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:18:58.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librbd1 2023-01-25T12:18:58.921 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:18:59.740 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:17 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:01.002 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:19:01.108 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:01.110 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:19:01.110 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:01.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rados 2023-01-25T12:19:01.592 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:02.380 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:20 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:03.657 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:19:03.765 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:03.766 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:19:03.767 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:03.905 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rgw 2023-01-25T12:19:04.257 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:05.022 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:06.298 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:19:06.423 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:06.424 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:19:06.424 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:06.538 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-cephfs 2023-01-25T12:19:06.891 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:07.700 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:25 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:09.049 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:19:09.159 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:19:09.160 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:09.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rbd 2023-01-25T12:19:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:10.432 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:28 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:11.704 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T12:19:11.813 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:11.815 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2023-01-25T12:19:11.815 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:11.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-fuse 2023-01-25T12:19:12.279 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:31 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:14.433 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:14.435 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:14.435 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2023-01-25T12:19:14.435 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:14.435 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:19:14.435 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T12:19:14.436 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:14.436 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:19:14.436 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:14.436 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:19:14.437 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:14.437 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 94 k 2023-01-25T12:19:14.437 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 280 k 2023-01-25T12:19:14.437 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:19:14.642 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 456 kB/s | 94 kB 00:00 2023-01-25T12:19:14.643 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:19:14.643 INFO:teuthology.orchestra.run.smithi090.stdout:Total 453 kB/s | 94 kB 00:00 2023-01-25T12:19:14.643 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:19:14.653 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:19:14.654 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:19:14.708 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:19:14.711 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:19:14.822 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:19:14.989 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:15.288 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:16.259 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:16.260 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:34 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:18.755 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:19:18.933 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:18.934 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:19:18.934 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:19:18.934 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:18.934 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:19.073 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-mirror 2023-01-25T12:19:19.426 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:20.256 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:38 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:21.664 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:21.665 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:21.665 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2023-01-25T12:19:21.666 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:21.666 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:19:21.666 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T12:19:21.666 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:21.666 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.5 M 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 16 M 2023-01-25T12:19:21.667 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:19:22.498 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 4.1 MB/s | 3.5 MB 00:00 2023-01-25T12:19:22.499 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:19:22.499 INFO:teuthology.orchestra.run.smithi090.stdout:Total 4.1 MB/s | 3.5 MB 00:00 2023-01-25T12:19:22.500 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:19:22.511 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:19:22.512 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:19:22.577 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:19:22.581 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:19:23.051 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:19:23.249 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:23.608 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:24.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:24.629 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:42 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:26.974 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:19:27.154 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:27.154 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:19:27.154 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:19:27.154 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:27.155 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:27.283 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-nbd 2023-01-25T12:19:27.638 INFO:teuthology.orchestra.run.smithi090.stdout:Updating Subscription Management repositories. 2023-01-25T12:19:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:46 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:29.879 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2023-01-25T12:19:29.880 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:29.880 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2023-01-25T12:19:29.881 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:29.881 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2023-01-25T12:19:29.881 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T12:19:29.881 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:29.881 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2023-01-25T12:19:29.882 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2023-01-25T12:19:29.882 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2023-01-25T12:19:29.882 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:29.882 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 180 k 2023-01-25T12:19:29.882 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 568 k 2023-01-25T12:19:29.883 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2023-01-25T12:19:30.165 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 632 kB/s | 180 kB 00:00 2023-01-25T12:19:30.166 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2023-01-25T12:19:30.166 INFO:teuthology.orchestra.run.smithi090.stdout:Total 629 kB/s | 180 kB 00:00 2023-01-25T12:19:30.166 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2023-01-25T12:19:30.177 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2023-01-25T12:19:30.178 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2023-01-25T12:19:30.231 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2023-01-25T12:19:30.235 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2023-01-25T12:19:30.347 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2023-01-25T12:19:30.496 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:30.746 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:31.752 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T12:19:31.752 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:03:49 ago on Wed 25 Jan 2023 12:15:42 PM UTC. 2023-01-25T12:19:34.064 INFO:teuthology.orchestra.run.smithi090.stdout:Installed products updated. 2023-01-25T12:19:34.246 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:34.247 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2023-01-25T12:19:34.247 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T12:19:34.247 INFO:teuthology.orchestra.run.smithi090.stdout: 2023-01-25T12:19:34.247 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2023-01-25T12:19:34.372 DEBUG:teuthology.parallel:result is None 2023-01-25T12:19:34.373 DEBUG:teuthology.orchestra.run.smithi090:> sudo lsb_release -is 2023-01-25T12:19:34.413 INFO:teuthology.orchestra.run.smithi090.stdout:RedHatEnterprise 2023-01-25T12:19:34.415 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T12:19:34.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:19:34.566 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T12:19:34.588 INFO:teuthology.orchestra.run.smithi090.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T12:19:34.589 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T12:19:34.589 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T12:19:34.589 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T12:19:34.590 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:19:34.590 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T12:19:34.655 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T12:19:34.656 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:19:34.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T12:19:34.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T12:19:34.787 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T12:19:34.787 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:19:34.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T12:19:34.852 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T12:19:34.918 INFO:teuthology.run_tasks:Running task workunit... 2023-01-25T12:19:34.932 INFO:tasks.workunit:Pulling workunits from ref 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:19:34.933 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-01-25T12:19:34.933 INFO:tasks.workunit:timeout=3h 2023-01-25T12:19:34.933 INFO:tasks.workunit:cleanup=True 2023-01-25T12:19:34.934 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2023-01-25T12:19:34.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T12:19:34.975 INFO:teuthology.orchestra.run.smithi090.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-01-25T12:19:34.975 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-01-25T12:19:35.031 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-01-25T12:19:35.032 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-01-25T12:19:35.088 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T12:19:35.144 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-01-25T12:20:15.937 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'. 2023-01-25T12:20:15.937 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.937 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-01-25T12:20:15.937 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2023-01-25T12:20:15.938 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2023-01-25T12:20:15.938 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.938 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2023-01-25T12:20:15.938 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2023-01-25T12:20:15.938 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.939 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2023-01-25T12:20:15.939 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.939 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2023-01-25T12:20:15.939 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.939 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2023-01-25T12:20:15.940 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.940 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-01-25T12:20:15.940 INFO:tasks.workunit.client.0.smithi090.stderr: 2023-01-25T12:20:15.940 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 44e4095da12 Merge branch 'wip-nitzan-filestore-removal' of https://github.com/NitzanMordhai/ceph into wip-yuri6-testing-2023-01-24-0746 2023-01-25T12:20:15.941 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-01-25T12:20:15.999 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2023-01-25T12:20:15.999 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-01-25T12:20:16.055 INFO:tasks.workunit:Running workunits matching misc on client.0... 2023-01-25T12:20:16.055 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2023-01-25T12:20:16.056 DEBUG:teuthology.orchestra.run.smithi090:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=44e4095da12a8269eb6cfe163cbb0d00b1a6412c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2023-01-25T12:20:16.134 INFO:tasks.workunit.client.0.smithi090.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-01-25T12:20:16.142 INFO:tasks.workunit.client.0.smithi090.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2023-01-25T12:20:16.142 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2023-01-25T12:20:16.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2023-01-25T12:20:16.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: main: export PYTHONWARNINGS=ignore 2023-01-25T12:20:16.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: main: PYTHONWARNINGS=ignore 2023-01-25T12:20:16.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: main: export CEPH_CONF=/dev/null 2023-01-25T12:20:16.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: main: CEPH_CONF=/dev/null 2023-01-25T12:20:16.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2240: main: unset CEPH_ARGS 2023-01-25T12:20:16.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2242: main: local code 2023-01-25T12:20:16.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2243: main: run td/mclock-config 2023-01-25T12:20:16.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2023-01-25T12:20:16.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2023-01-25T12:20:16.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2023-01-25T12:20:16.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2023-01-25T12:20:16.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2023-01-25T12:20:16.145 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2023-01-25T12:20:16.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none ' 2023-01-25T12:20:16.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2023-01-25T12:20:16.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2023-01-25T12:20:16.146 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2023-01-25T12:20:16.146 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2023-01-25T12:20:16.148 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2023-01-25T12:20:16.149 INFO:tasks.workunit.client.0.smithi090.stderr:TEST_profile_builtin_to_custom 2023-01-25T12:20:16.149 INFO:tasks.workunit.client.0.smithi090.stderr:TEST_profile_custom_to_builtin 2023-01-25T12:20:16.149 INFO:tasks.workunit.client.0.smithi090.stderr:TEST_recovery_limit_adjustment_mclock' 2023-01-25T12:20:16.149 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:20:16.150 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:20:16.151 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:20:16.151 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:20:16.151 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:20:16.151 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:20:16.151 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:20:16.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:20:16.152 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:20:16.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:20:16.152 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:20:16.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:20:16.153 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:20:16.153 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:20:16.155 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:16.155 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:20:16.155 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:20:16.155 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:16.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:20:16.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:20:16.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:20:16.156 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:20:16.156 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.157 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:20:16.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:20:16.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:20:16.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:20:16.158 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:20:16.158 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.158 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.158 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:20:16.161 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:20:16.161 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2023-01-25T12:20:16.161 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2023-01-25T12:20:16.161 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:20:16.162 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:20:16.162 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2023-01-25T12:20:16.162 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:228: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2023-01-25T12:20:16.162 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2023-01-25T12:20:16.162 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:20:16.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:20:16.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:20:16.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:20:16.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:20:16.163 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:20:16.164 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:20:16.164 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:20:16.164 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:20:16.164 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:20:16.164 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:20:16.165 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:20:16.165 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:20:16.165 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:20:16.165 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:20:16.165 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:20:16.166 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:20:16.169 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:16.169 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:20:16.169 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:20:16.169 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:20:16.169 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:20:16.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:20:16.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:20:16.170 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:20:16.170 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.170 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:20:16.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:20:16.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:20:16.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:20:16.171 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:20:16.172 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.172 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:20:16.172 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:20:16.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:20:16.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:20:16.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:20:16.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:231: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2023-01-25T12:20:16.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/mclock-config 2023-01-25T12:20:16.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-01-25T12:20:16.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-01-25T12:20:16.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-01-25T12:20:16.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/mclock-config/a 2023-01-25T12:20:16.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2023-01-25T12:20:16.662 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-01-25T12:20:16.662 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:20:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2023-01-25T12:20:16.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-01-25T12:20:16.957 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-01-25T12:20:16.957 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:20:16.958 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:20:16.958 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=fsid 2023-01-25T12:20:16.958 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .fsid 2023-01-25T12:20:16.958 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:20:16.958 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:20:16.959 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:20:16.959 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:20:16.959 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:16.959 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:16.959 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:20:16.960 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:20:16.960 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get fsid 2023-01-25T12:20:17.059 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-01-25T12:20:17.059 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:20:17.059 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:20:17.059 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=mon_host 2023-01-25T12:20:17.060 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .mon_host 2023-01-25T12:20:17.060 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:20:17.060 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:20:17.060 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:20:17.060 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:20:17.061 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:17.061 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:17.061 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:20:17.061 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:20:17.061 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get mon_host 2023-01-25T12:20:17.159 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2023-01-25T12:20:17.159 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/mclock-config 2023-01-25T12:20:17.159 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-01-25T12:20:17.159 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-01-25T12:20:17.160 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-01-25T12:20:17.160 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/mclock-config/x 2023-01-25T12:20:17.160 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-01-25T12:20:18.038 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-01-25T12:20:18.039 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:20:18.039 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:20:18.039 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:20:18.039 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:18.040 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:18.040 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:20:18.040 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:20:18.045 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:20:18.074 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:234: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2023-01-25T12:20:18.075 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:20:18.075 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:20:18.075 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-01-25T12:20:18.075 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:20:18.075 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/0 2023-01-25T12:20:18.076 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:20:18.076 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:20:18.076 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:20:18.076 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:20:18.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2023-01-25T12:20:18.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2023-01-25T12:20:18.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:20:18.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:20:18.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:20:18.078 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:20:18.078 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:20:18.078 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:20:18.078 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:20:18.078 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:20:18.079 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:20:18.079 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:20:18.079 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:20:18.079 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:20:18.080 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:20:18.080 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:20:18.080 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:20:18.080 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:20:18.080 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:20:18.081 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:20:18.081 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:20:18.081 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:20:18.082 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:20:18.082 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/0 2023-01-25T12:20:18.082 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:20:18.082 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=e6385c9c-09f0-4407-b4e6-c1632731022f 2023-01-25T12:20:18.082 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd0 e6385c9c-09f0-4407-b4e6-c1632731022f' 2023-01-25T12:20:18.083 INFO:tasks.workunit.client.0.smithi090.stdout:add osd0 e6385c9c-09f0-4407-b4e6-c1632731022f 2023-01-25T12:20:18.083 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:20:18.092 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQCCHtFjerBjBRAA3URZiUXREVQGmxxVvtXCVg== 2023-01-25T12:20:18.092 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQCCHtFjerBjBRAA3URZiUXREVQGmxxVvtXCVg=="}' 2023-01-25T12:20:18.092 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new e6385c9c-09f0-4407-b4e6-c1632731022f -i td/mclock-config/0/new.json 2023-01-25T12:20:18.467 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:20:18.478 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/0/new.json 2023-01-25T12:20:18.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQCCHtFjerBjBRAA3URZiUXREVQGmxxVvtXCVg== --osd-uuid e6385c9c-09f0-4407-b4e6-c1632731022f 2023-01-25T12:20:18.506 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.504+0000 7f80f448f580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:18.508 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.506+0000 7f80f448f580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:18.509 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.507+0000 7f80f448f580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:18.509 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.508+0000 7f80f448f580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:20:18.510 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.508+0000 7f80f448f580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:20:18.510 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.508+0000 7f80f448f580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:20:18.584 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:18.582+0000 7f80f448f580 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2023-01-25T12:20:24.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/0/keyring 2023-01-25T12:20:24.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:20:24.422 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd0 key to auth repository 2023-01-25T12:20:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd0 key to auth repository 2023-01-25T12:20:24.422 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:20:24.678 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.0 2023-01-25T12:20:24.678 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.0 2023-01-25T12:20:24.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:20:24.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:20:24.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:20:24.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:20:24.703 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:24.701+0000 7f3188485580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:24.705 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:24.704+0000 7f3188485580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:24.706 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:24.705+0000 7f3188485580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:20:24.927 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 0 2023-01-25T12:20:24.927 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:20:24.927 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=0 2023-01-25T12:20:24.928 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:20:24.928 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:20:24.928 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:24.928 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:20:24.929 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:20:24.929 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:24.929 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:25.165 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:25.232 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:25.231+0000 7f3188485580 -1 Falling back to public interface 2023-01-25T12:20:26.166 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:26.167 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:26.167 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:20:26.168 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:20:26.168 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:26.168 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:26.611 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:27.613 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:27.613 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:27.613 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:20:27.613 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:20:27.614 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:27.614 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:27.720 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:20:27.719+0000 7f3188485580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:20:28.135 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:29.137 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:29.137 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:29.137 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:20:29.138 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:20:29.138 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:29.139 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:29.570 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:30.571 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:30.571 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:30.572 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:20:30.572 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:20:30.573 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:30.573 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:31.003 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:32.004 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:32.004 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:32.005 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:20:32.005 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:20:32.005 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:32.006 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:32.492 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:33.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:33.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:33.494 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:20:33.494 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:20:33.495 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:33.495 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:33.944 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:34.946 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:34.947 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:34.948 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:20:34.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:20:34.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:34.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:35.483 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:36.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:36.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:36.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:20:36.485 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:20:36.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:36.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:36.933 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:37.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:37.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:37.935 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:20:37.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:20:37.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:37.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:38.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:39.489 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:39.490 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:39.490 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:20:39.490 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:20:39.491 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:39.491 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:40.622 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:41.624 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:41.624 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:41.624 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:20:41.625 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:20:41.625 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:41.626 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:43.482 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:44.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:44.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:44.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:20:44.486 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:20:44.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:44.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:45.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:46.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:46.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:46.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:20:46.147 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:20:46.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:46.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:46.736 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:47.738 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:47.738 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:47.738 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:20:47.738 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:20:47.739 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:47.739 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:48.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:49.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:49.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:49.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:20:49.287 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:20:49.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:49.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:50.862 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:51.863 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:51.863 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:51.863 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:20:51.864 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:20:51.864 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:51.865 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:53.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:54.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:54.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:54.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:20:54.674 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:20:54.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:54.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:56.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:57.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:57.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:57.661 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:20:57.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:20:57.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:20:57.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:20:58.996 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:20:59.997 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:20:59.998 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:20:59.998 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:20:59.999 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:20:59.999 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:00.000 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:00.436 INFO:tasks.workunit.client.0.smithi090.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/87439,v1:127.0.0.1:6803/87439] [v2:127.0.0.1:6804/87439,v1:127.0.0.1:6805/87439] exists,up e6385c9c-09f0-4407-b4e6-c1632731022f 2023-01-25T12:21:00.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:21:00.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:21:00.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:21:00.439 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:21:00.439 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:21:00.440 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:21:00.440 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:21:00.440 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:00.441 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:00.441 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:21:00.441 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:21:00.442 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_max_backfills 2023-01-25T12:21:00.545 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local 'backfills={"osd_max_backfills":"10"}' 2023-01-25T12:21:00.545 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: echo '{"osd_max_backfills":"10"}' 2023-01-25T12:21:00.546 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: grep --quiet osd_max_backfills 2023-01-25T12:21:00.546 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:243: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2023-01-25T12:21:01.148 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: sleep 2 2023-01-25T12:21:03.150 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:21:03.150 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:21:03.151 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:21:03.151 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:21:03.151 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:03.151 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:03.151 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:21:03.152 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:21:03.152 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_max_backfills 2023-01-25T12:21:03.269 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: local 'max_backfills={"osd_max_backfills":"10"}' 2023-01-25T12:21:03.269 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:247: TEST_backfill_limit_adjustment_mclock: test '{"osd_max_backfills":"10"}' = '{"osd_max_backfills":"10"}' 2023-01-25T12:21:03.270 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2023-01-25T12:21:03.748 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2023-01-25T12:21:04.234 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:253: TEST_backfill_limit_adjustment_mclock: sleep 2 2023-01-25T12:21:06.236 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:21:06.237 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:21:06.237 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:21:06.238 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_max_backfills 2023-01-25T12:21:06.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: max_backfills='{"osd_max_backfills":"20"}' 2023-01-25T12:21:06.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:256: TEST_backfill_limit_adjustment_mclock: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2023-01-25T12:21:06.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:258: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2023-01-25T12:21:06.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:21:06.356 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:21:06.356 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:21:06.356 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:21:06.356 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:21:06.356 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:21:06.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:21:06.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:21:06.460 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:21:06.460 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:21:06.461 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:21:06.461 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:21:06.462 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:21:06.462 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:21:06.463 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:21:06.464 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.464 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:21:06.465 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:21:06.465 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.465 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:21:06.466 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:21:06.466 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:21:06.471 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:21:06.471 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.471 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.472 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:21:06.473 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:21:06.473 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:21:06.475 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2023-01-25T12:21:06.475 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:21:06.476 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:21:06.476 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:21:06.476 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:21:06.476 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:21:06.476 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:21:06.477 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:21:06.477 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:21:06.478 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:21:06.478 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:21:06.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:21:06.479 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:21:06.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:21:06.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:21:06.480 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:21:06.480 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.481 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:21:06.481 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:21:06.481 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:21:06.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:21:06.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:21:06.485 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:21:06.485 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.485 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:21:06.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:21:06.487 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:21:06.487 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:21:06.487 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:21:06.487 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:21:06.487 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:21:06.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:21:06.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:21:06.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:21:06.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:21:06.488 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:21:06.490 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:21:06.490 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:21:06.490 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:21:06.490 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:21:06.491 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:21:06.491 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.491 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:21:06.492 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:21:06.492 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.492 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:21:06.492 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:21:06.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:21:06.495 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:21:06.496 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.496 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.496 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:21:06.496 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:21:06.496 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:21:06.497 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:21:06.497 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:21:06.497 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.497 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.497 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:21:06.498 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:21:06.498 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:21:06.498 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:21:06.498 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:21:06.498 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:40: TEST_profile_builtin_to_custom: local OSDS=3 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: setup td/mclock-config 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:21:06.499 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:21:06.500 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:21:06.500 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:21:06.500 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:21:06.500 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:21:06.500 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:21:06.501 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:21:06.501 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:21:06.503 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:21:06.504 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:21:06.505 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:21:06.505 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:21:06.505 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:21:06.506 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:21:06.506 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:21:06.506 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.506 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:21:06.507 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:21:06.507 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:21:06.507 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:21:06.507 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:21:06.507 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:21:06.508 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:21:06.508 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.508 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.508 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:21:06.511 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:21:06.511 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:21:06.512 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:21:06.512 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:21:06.512 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.512 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:21:06.514 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:21:06.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:21:06.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:21:06.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:21:06.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_mon td/mclock-config a --osd_pool_default_size=3 2023-01-25T12:21:06.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/mclock-config 2023-01-25T12:21:06.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-01-25T12:21:06.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-01-25T12:21:06.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-01-25T12:21:06.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/mclock-config/a 2023-01-25T12:21:06.516 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config --osd_pool_default_size=3 2023-01-25T12:21:06.889 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-01-25T12:21:06.890 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:21:06.890 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:21:06.890 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:21:06.890 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:06.891 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:06.891 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:06.891 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2023-01-25T12:21:07.133 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-01-25T12:21:07.135 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-01-25T12:21:07.135 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:21:07.135 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:21:07.135 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=fsid 2023-01-25T12:21:07.136 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .fsid 2023-01-25T12:21:07.136 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:21:07.136 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:21:07.136 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:21:07.136 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:21:07.137 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:07.137 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:07.137 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:21:07.137 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:21:07.138 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get fsid 2023-01-25T12:21:07.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-01-25T12:21:07.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:21:07.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:21:07.242 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=mon_host 2023-01-25T12:21:07.242 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .mon_host 2023-01-25T12:21:07.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:21:07.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:21:07.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:21:07.243 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:21:07.243 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:07.243 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:07.243 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:21:07.243 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:21:07.244 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get mon_host 2023-01-25T12:21:07.341 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:44: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2023-01-25T12:21:07.341 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/mclock-config 2023-01-25T12:21:07.342 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-01-25T12:21:07.342 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-01-25T12:21:07.342 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-01-25T12:21:07.342 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/mclock-config/x 2023-01-25T12:21:07.342 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-01-25T12:21:08.240 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-01-25T12:21:08.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:21:08.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:21:08.241 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:21:08.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:08.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:08.242 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:08.242 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:21:08.243 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:21:08.268 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:45: TEST_profile_builtin_to_custom: expr 3 - 1 2023-01-25T12:21:08.269 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:45: TEST_profile_builtin_to_custom: seq 0 2 2023-01-25T12:21:08.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:45: TEST_profile_builtin_to_custom: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:21:08.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2023-01-25T12:21:08.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:21:08.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:21:08.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-01-25T12:21:08.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:21:08.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/0 2023-01-25T12:21:08.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:21:08.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:21:08.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:21:08.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:21:08.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2023-01-25T12:21:08.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2023-01-25T12:21:08.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:21:08.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:21:08.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:21:08.283 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:21:08.284 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:21:08.284 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:21:08.284 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:21:08.284 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:08.284 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:21:08.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:21:08.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:21:08.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:21:08.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:21:08.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:21:08.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:21:08.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:21:08.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/0 2023-01-25T12:21:08.287 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:21:08.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=bf85c615-29cd-48f0-818e-d1c2df41cf3e 2023-01-25T12:21:08.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd0 bf85c615-29cd-48f0-818e-d1c2df41cf3e' 2023-01-25T12:21:08.288 INFO:tasks.workunit.client.0.smithi090.stdout:add osd0 bf85c615-29cd-48f0-818e-d1c2df41cf3e 2023-01-25T12:21:08.289 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:21:08.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQC0HtFj0cGxERAAQ50nsBmIxM07n+4xf/6TPg== 2023-01-25T12:21:08.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQC0HtFj0cGxERAAQ50nsBmIxM07n+4xf/6TPg=="}' 2023-01-25T12:21:08.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new bf85c615-29cd-48f0-818e-d1c2df41cf3e -i td/mclock-config/0/new.json 2023-01-25T12:21:08.613 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:21:08.624 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/0/new.json 2023-01-25T12:21:08.625 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQC0HtFj0cGxERAAQ50nsBmIxM07n+4xf/6TPg== --osd-uuid bf85c615-29cd-48f0-818e-d1c2df41cf3e 2023-01-25T12:21:08.652 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.650+0000 7f932f9bd580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:08.679 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.677+0000 7f932f9bd580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:08.679 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.678+0000 7f932f9bd580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:08.680 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.678+0000 7f932f9bd580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:21:08.681 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.678+0000 7f932f9bd580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:21:08.681 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.678+0000 7f932f9bd580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:21:08.719 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:08.717+0000 7f932f9bd580 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2023-01-25T12:21:14.008 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/0/keyring 2023-01-25T12:21:14.008 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:21:14.009 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd0 key to auth repository 2023-01-25T12:21:14.009 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd0 key to auth repository 2023-01-25T12:21:14.009 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:21:14.292 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.0 2023-01-25T12:21:14.293 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.0 2023-01-25T12:21:14.293 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:21:14.294 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:21:14.294 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:21:14.294 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:21:14.317 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:14.315+0000 7fa25700b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:14.344 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:14.341+0000 7fa25700b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:14.345 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:14.343+0000 7fa25700b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:14.723 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 0 2023-01-25T12:21:14.723 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:21:14.723 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=0 2023-01-25T12:21:14.723 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:21:14.723 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:21:14.724 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:14.724 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:21:14.724 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:21:14.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:14.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:14.876 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:14.875+0000 7fa25700b580 -1 Falling back to public interface 2023-01-25T12:21:15.190 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:16.192 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:16.192 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:16.193 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:21:16.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:21:16.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:16.194 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:16.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:17.295 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:17.293+0000 7fa25700b580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:21:17.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:17.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:17.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:21:17.662 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:21:17.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:17.664 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:18.195 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:19.196 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:19.196 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:19.197 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:21:19.197 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:21:19.198 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:19.198 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:19.645 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:20.646 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:20.647 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:20.647 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:21:20.647 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:21:20.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:20.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:21.195 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:22.197 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:22.197 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:22.198 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:21:22.198 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:21:22.200 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:22.200 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:22.631 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:23.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:23.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:23.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:21:23.633 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:21:23.634 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:23.634 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:24.188 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:25.189 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:25.190 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:25.190 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:21:25.190 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:21:25.191 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:25.191 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:25.624 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:26.626 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:26.626 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:26.626 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:21:26.627 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:21:26.627 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:26.627 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:27.099 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:28.100 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:28.101 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:28.101 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:21:28.101 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:21:28.102 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:28.102 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:28.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:29.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:29.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:29.534 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:21:29.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:21:29.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:29.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:30.151 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:31.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:31.153 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:31.153 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:21:31.153 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:21:31.154 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:31.154 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:33.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:34.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:34.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:34.019 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:21:34.019 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:21:34.019 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:34.020 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:35.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:36.643 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:36.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:36.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:21:36.644 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:21:36.645 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:36.645 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:37.223 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:38.224 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:38.224 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:38.225 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:21:38.225 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:21:38.225 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:38.226 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:39.045 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:40.047 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:40.047 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:40.047 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:21:40.048 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:21:40.048 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:40.049 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:41.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:42.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:42.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:42.360 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:21:42.360 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:21:42.361 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:42.361 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:43.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:44.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:44.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:44.329 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:21:44.329 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:21:44.329 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:44.330 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:45.915 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:46.916 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:46.917 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:46.917 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:21:46.917 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:21:46.918 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:46.918 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:21:47.347 INFO:tasks.workunit.client.0.smithi090.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/89821,v1:127.0.0.1:6803/89821] [v2:127.0.0.1:6804/89821,v1:127.0.0.1:6805/89821] exists,up bf85c615-29cd-48f0-818e-d1c2df41cf3e 2023-01-25T12:21:47.351 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:21:47.352 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:21:47.352 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:21:47.352 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:45: TEST_profile_builtin_to_custom: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:21:47.352 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: run_osd td/mclock-config 1 --osd_op_queue=mclock_scheduler 2023-01-25T12:21:47.352 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:21:47.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:21:47.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-01-25T12:21:47.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:21:47.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/1 2023-01-25T12:21:47.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:21:47.354 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:21:47.354 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:21:47.354 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:21:47.354 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/1' 2023-01-25T12:21:47.354 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/1/journal' 2023-01-25T12:21:47.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:21:47.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:21:47.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:21:47.355 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:21:47.355 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:21:47.356 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:21:47.356 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:21:47.356 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:21:47.356 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:21:47.356 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:47.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:21:47.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:21:47.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:21:47.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:21:47.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:21:47.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:21:47.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:21:47.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:21:47.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:21:47.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:21:47.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:21:47.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/1 2023-01-25T12:21:47.359 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:21:47.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=4ab99fe1-f439-427e-b094-199649ca6936 2023-01-25T12:21:47.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd1 4ab99fe1-f439-427e-b094-199649ca6936' 2023-01-25T12:21:47.360 INFO:tasks.workunit.client.0.smithi090.stdout:add osd1 4ab99fe1-f439-427e-b094-199649ca6936 2023-01-25T12:21:47.360 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:21:47.368 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQDbHtFj56LaFRAAgdPvA/k9U2mOqbDAskUd5w== 2023-01-25T12:21:47.368 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQDbHtFj56LaFRAAgdPvA/k9U2mOqbDAskUd5w=="}' 2023-01-25T12:21:47.368 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 4ab99fe1-f439-427e-b094-199649ca6936 -i td/mclock-config/1/new.json 2023-01-25T12:21:47.872 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:21:47.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/1/new.json 2023-01-25T12:21:47.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 1 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/1 --osd-journal=td/mclock-config/1/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQDbHtFj56LaFRAAgdPvA/k9U2mOqbDAskUd5w== --osd-uuid 4ab99fe1-f439-427e-b094-199649ca6936 2023-01-25T12:21:47.910 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.910+0000 7efdbbac2580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:47.912 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.912+0000 7efdbbac2580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:47.913 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.913+0000 7efdbbac2580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:47.914 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.913+0000 7efdbbac2580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:21:47.914 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.913+0000 7efdbbac2580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:21:47.914 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.913+0000 7efdbbac2580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:21:47.969 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:47.968+0000 7efdbbac2580 -1 bluestore(td/mclock-config/1) _read_fsid unparsable uuid 2023-01-25T12:21:54.050 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/1/keyring 2023-01-25T12:21:54.051 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:21:54.051 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd1 key to auth repository 2023-01-25T12:21:54.052 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd1 key to auth repository 2023-01-25T12:21:54.052 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:21:54.647 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.1 2023-01-25T12:21:54.647 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.1 2023-01-25T12:21:54.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 1 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/1 --osd-journal=td/mclock-config/1/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:21:54.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:21:54.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:21:54.649 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:21:54.671 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:54.669+0000 7f6fa4660580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:54.676 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:54.674+0000 7f6fa4660580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:54.677 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:54.675+0000 7f6fa4660580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:21:55.115 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 1 2023-01-25T12:21:55.115 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:21:55.115 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=1 2023-01-25T12:21:55.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:21:55.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:21:55.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:55.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:21:55.117 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:21:55.117 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:55.117 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:21:55.213 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:55.211+0000 7f6fa4660580 -1 Falling back to public interface 2023-01-25T12:21:55.578 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:56.580 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:56.580 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:56.580 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:21:56.581 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:21:56.581 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:56.581 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:21:57.015 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:57.591 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:21:57.589+0000 7f6fa4660580 -1 osd.1 0 log_to_monitors true 2023-01-25T12:21:58.016 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:58.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:58.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:21:58.017 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:21:58.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:58.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:21:58.512 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:21:59.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:21:59.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:21:59.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:21:59.514 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:21:59.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:21:59.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:21:59.952 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:00.953 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:00.954 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:00.954 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:22:00.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:22:00.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:00.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:01.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:02.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:02.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:02.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:22:02.535 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:22:02.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:02.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:03.028 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:04.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:04.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:04.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:22:04.031 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:22:04.031 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:04.031 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:04.507 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:05.509 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:05.509 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:05.510 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:22:05.510 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:22:05.511 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:05.511 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:05.946 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:06.947 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:06.948 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:06.948 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:22:06.948 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:22:06.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:06.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:07.528 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:08.530 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:08.530 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:08.530 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:22:08.531 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:22:08.531 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:08.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:09.003 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:10.004 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:10.005 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:10.005 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:22:10.006 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:22:10.006 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:10.006 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:11.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:12.533 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:12.533 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:12.534 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:22:12.534 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:22:12.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:12.535 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:13.151 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:14.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:14.153 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:14.153 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:22:14.153 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:22:14.154 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:14.154 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:15.710 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:16.711 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:16.711 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:16.712 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:22:16.712 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:22:16.713 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:16.713 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:17.639 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:18.640 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:18.641 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:18.641 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:22:18.641 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:22:18.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:18.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:20.040 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:21.042 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:21.042 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:21.042 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:22:21.043 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:22:21.043 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:21.044 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:21.519 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:22.521 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:22.521 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:22.521 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:22:22.522 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:22:22.522 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:22.522 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:23.394 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:24.395 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:24.395 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:24.396 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:22:24.396 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:22:24.397 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:24.397 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:25.861 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:26.862 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:26.863 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:26.863 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:22:26.864 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:22:26.864 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:26.864 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:27.343 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:28.344 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:28.344 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:28.344 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:22:28.345 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:22:28.345 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:28.346 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:22:28.778 INFO:tasks.workunit.client.0.smithi090.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/91614,v1:127.0.0.1:6811/91614] [v2:127.0.0.1:6812/91614,v1:127.0.0.1:6813/91614] exists,up 4ab99fe1-f439-427e-b094-199649ca6936 2023-01-25T12:22:28.782 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:22:28.782 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:22:28.783 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:22:28.783 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:45: TEST_profile_builtin_to_custom: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:22:28.783 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: run_osd td/mclock-config 2 --osd_op_queue=mclock_scheduler 2023-01-25T12:22:28.783 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:22:28.784 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:22:28.784 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-01-25T12:22:28.784 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:22:28.784 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/2 2023-01-25T12:22:28.784 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:22:28.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:22:28.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:22:28.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:22:28.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/2' 2023-01-25T12:22:28.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/2/journal' 2023-01-25T12:22:28.786 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:22:28.786 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:22:28.786 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:22:28.786 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:22:28.786 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:22:28.787 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:22:28.787 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:22:28.787 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:22:28.787 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:22:28.787 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:22:28.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:22:28.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:22:28.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:22:28.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:22:28.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:22:28.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:22:28.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:22:28.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:22:28.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:22:28.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:22:28.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:22:28.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/2 2023-01-25T12:22:28.790 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:22:28.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=9f9a584f-8f35-457c-8963-c14310edf840 2023-01-25T12:22:28.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd2 9f9a584f-8f35-457c-8963-c14310edf840' 2023-01-25T12:22:28.791 INFO:tasks.workunit.client.0.smithi090.stdout:add osd2 9f9a584f-8f35-457c-8963-c14310edf840 2023-01-25T12:22:28.791 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:22:28.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQAEH9FjTCmMLxAAuNoyakjhImvRROFyZycR2w== 2023-01-25T12:22:28.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQAEH9FjTCmMLxAAuNoyakjhImvRROFyZycR2w=="}' 2023-01-25T12:22:28.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 9f9a584f-8f35-457c-8963-c14310edf840 -i td/mclock-config/2/new.json 2023-01-25T12:22:29.365 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:22:29.376 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/2/new.json 2023-01-25T12:22:29.377 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 2 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/2 --osd-journal=td/mclock-config/2/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQAEH9FjTCmMLxAAuNoyakjhImvRROFyZycR2w== --osd-uuid 9f9a584f-8f35-457c-8963-c14310edf840 2023-01-25T12:22:29.403 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.402+0000 7fc919da7580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:29.405 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.403+0000 7fc919da7580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:29.406 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.404+0000 7fc919da7580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:29.406 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.405+0000 7fc919da7580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:22:29.406 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.405+0000 7fc919da7580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:22:29.407 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.405+0000 7fc919da7580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:22:29.480 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:29.479+0000 7fc919da7580 -1 bluestore(td/mclock-config/2) _read_fsid unparsable uuid 2023-01-25T12:22:35.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/2/keyring 2023-01-25T12:22:35.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:22:35.645 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd2 key to auth repository 2023-01-25T12:22:35.645 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd2 key to auth repository 2023-01-25T12:22:35.645 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:22:36.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.2 2023-01-25T12:22:36.234 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.2 2023-01-25T12:22:36.235 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:22:36.235 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 2 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/2 --osd-journal=td/mclock-config/2/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:22:36.235 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:22:36.235 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:22:36.258 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:36.257+0000 7f5ff242d580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:36.288 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:36.286+0000 7f5ff242d580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:36.289 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:36.288+0000 7f5ff242d580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:22:36.686 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 2 2023-01-25T12:22:36.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:22:36.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=2 2023-01-25T12:22:36.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:22:36.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:22:36.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:36.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:22:36.688 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:22:36.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:36.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:36.824 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:36.823+0000 7f5ff242d580 -1 Falling back to public interface 2023-01-25T12:22:37.154 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:38.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:38.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:38.157 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:22:38.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:22:38.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:38.158 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:38.593 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:39.227 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:22:39.225+0000 7f5ff242d580 -1 osd.2 0 log_to_monitors true 2023-01-25T12:22:39.595 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:39.595 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:39.595 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:22:39.596 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:22:39.596 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:39.596 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:40.106 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:41.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:41.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:41.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:22:41.109 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:22:41.109 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:41.110 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:41.554 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:42.555 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:42.556 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:42.556 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:22:42.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:22:42.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:42.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:43.000 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:44.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:44.002 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:44.002 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:22:44.002 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:22:44.003 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:44.003 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:44.466 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:45.467 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:45.467 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:45.468 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:22:45.468 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:22:45.469 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:45.469 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:45.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:46.903 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:46.903 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:46.904 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:22:46.904 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:22:46.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:46.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:47.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:48.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:48.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:48.443 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:22:48.443 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:22:48.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:48.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:48.888 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:49.889 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:49.889 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:49.889 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:22:49.890 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:22:49.890 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:49.891 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:50.451 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:51.452 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:51.452 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:51.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:22:51.454 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:22:51.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:51.455 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:52.250 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:53.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:53.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:53.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:22:53.253 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:22:53.253 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:53.253 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:53.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:54.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:54.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:54.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:22:54.689 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:22:54.690 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:54.690 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:56.056 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:57.057 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:57.058 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:57.058 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:22:57.058 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:22:57.059 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:57.059 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:57.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:22:58.717 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:22:58.717 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:22:58.718 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:22:58.718 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:22:58.718 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:22:58.719 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:22:59.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:00.487 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:00.488 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:00.488 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:23:00.489 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:23:00.489 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:00.489 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:01.189 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:02.190 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:02.190 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:02.191 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:23:02.191 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:23:02.192 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:02.192 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:03.206 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:04.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:04.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:04.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:23:04.209 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:23:04.209 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:04.210 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:05.836 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:06.837 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:06.837 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:06.838 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:23:06.838 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:23:06.839 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:06.839 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:07.372 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:08.374 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:08.374 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:08.374 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:23:08.375 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:23:08.375 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:08.375 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:09.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:10.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:10.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:10.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 20 2023-01-25T12:23:10.327 INFO:tasks.workunit.client.0.smithi090.stdout:20 2023-01-25T12:23:10.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:10.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:23:10.826 INFO:tasks.workunit.client.0.smithi090.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/93425,v1:127.0.0.1:6819/93425] [v2:127.0.0.1:6820/93425,v1:127.0.0.1:6821/93425] exists,up 9f9a584f-8f35-457c-8963-c14310edf840 2023-01-25T12:23:10.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:23:10.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:23:10.828 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:23:10.828 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:51: TEST_profile_builtin_to_custom: expr 3 - 1 2023-01-25T12:23:10.829 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:51: TEST_profile_builtin_to_custom: seq 0 2 2023-01-25T12:23:10.829 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:51: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:10.829 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2023-01-25T12:23:11.319 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: local mclock_profile=high_client_ops 2023-01-25T12:23:11.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_client_ops = high_client_ops 2023-01-25T12:23:11.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:51: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:11.320 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: ceph config get osd.1 osd_mclock_profile 2023-01-25T12:23:11.814 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: local mclock_profile=high_client_ops 2023-01-25T12:23:11.815 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_client_ops = high_client_ops 2023-01-25T12:23:11.815 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:51: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:11.815 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: ceph config get osd.2 osd_mclock_profile 2023-01-25T12:23:12.321 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: local mclock_profile=high_client_ops 2023-01-25T12:23:12.322 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_client_ops = high_client_ops 2023-01-25T12:23:12.322 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:58: TEST_profile_builtin_to_custom: ceph config set osd osd_mclock_profile custom 2023-01-25T12:23:12.875 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: expr 3 - 1 2023-01-25T12:23:12.876 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: seq 0 2 2023-01-25T12:23:12.876 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:12.877 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2023-01-25T12:23:13.319 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: local mclock_profile=custom 2023-01-25T12:23:13.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:64: TEST_profile_builtin_to_custom: test custom = custom 2023-01-25T12:23:13.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:13.320 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: ceph config get osd.1 osd_mclock_profile 2023-01-25T12:23:13.753 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: local mclock_profile=custom 2023-01-25T12:23:13.754 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:64: TEST_profile_builtin_to_custom: test custom = custom 2023-01-25T12:23:13.754 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:13.754 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: ceph config get osd.2 osd_mclock_profile 2023-01-25T12:23:14.189 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: local mclock_profile=custom 2023-01-25T12:23:14.189 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:64: TEST_profile_builtin_to_custom: test custom = custom 2023-01-25T12:23:14.191 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2023-01-25T12:23:14.191 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc 2023-01-25T12:23:14.191 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: get_asok_path osd.2 2023-01-25T12:23:14.191 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:23:14.192 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:23:14.192 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:14.192 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:14.192 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:14.192 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:23:14.193 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: CEPH_ARGS= 2023-01-25T12:23:14.193 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:23:14.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res=17 2023-01-25T12:23:14.300 INFO:tasks.workunit.client.0.smithi090.stdout:client_res = 17 2023-01-25T12:23:14.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res = 17' 2023-01-25T12:23:14.300 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: expr 17 + 10 2023-01-25T12:23:14.301 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: local client_res_new=27 2023-01-25T12:23:14.301 INFO:tasks.workunit.client.0.smithi090.stdout:client_res_new = 27 2023-01-25T12:23:14.302 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:73: TEST_profile_builtin_to_custom: echo 'client_res_new = 27' 2023-01-25T12:23:14.302 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:74: TEST_profile_builtin_to_custom: ceph config set osd osd_mclock_scheduler_client_res 27 2023-01-25T12:23:14.810 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:76: TEST_profile_builtin_to_custom: expr 3 - 1 2023-01-25T12:23:14.811 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:76: TEST_profile_builtin_to_custom: seq 0 2 2023-01-25T12:23:14.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:76: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:14.812 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2023-01-25T12:23:15.293 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: local res=27 2023-01-25T12:23:15.293 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:81: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:15.295 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2023-01-25T12:23:15.295 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc 2023-01-25T12:23:15.295 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: get_asok_path osd.0 2023-01-25T12:23:15.295 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:23:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:23:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:15.296 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:15.297 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:23:15.297 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: CEPH_ARGS= 2023-01-25T12:23:15.297 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:23:15.402 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: res=27 2023-01-25T12:23:15.402 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:86: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:15.403 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:76: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:15.403 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: ceph config get osd.1 osd_mclock_scheduler_client_res 2023-01-25T12:23:15.853 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: local res=27 2023-01-25T12:23:15.853 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:81: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:15.854 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2023-01-25T12:23:15.855 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc 2023-01-25T12:23:15.855 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: get_asok_path osd.1 2023-01-25T12:23:15.855 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-01-25T12:23:15.855 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-01-25T12:23:15.855 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:15.856 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:15.856 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:15.856 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.1.asok 2023-01-25T12:23:15.856 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: CEPH_ARGS= 2023-01-25T12:23:15.856 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.1.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:23:15.961 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: res=27 2023-01-25T12:23:15.961 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:86: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:15.962 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:76: TEST_profile_builtin_to_custom: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:15.962 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: ceph config get osd.2 osd_mclock_scheduler_client_res 2023-01-25T12:23:16.422 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:80: TEST_profile_builtin_to_custom: local res=27 2023-01-25T12:23:16.423 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:81: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:16.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2023-01-25T12:23:16.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc 2023-01-25T12:23:16.423 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: get_asok_path osd.2 2023-01-25T12:23:16.423 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:23:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:23:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.424 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:23:16.425 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: CEPH_ARGS= 2023-01-25T12:23:16.425 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:23:16.530 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: res=27 2023-01-25T12:23:16.530 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:86: TEST_profile_builtin_to_custom: test 27 -eq 27 2023-01-25T12:23:16.531 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2023-01-25T12:23:16.531 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:23:16.531 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:23:16.531 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:23:16.531 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:23:16.532 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:23:16.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:23:16.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:23:16.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:23:16.637 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:23:16.638 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:23:16.639 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:23:16.639 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:23:16.640 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:23:16.640 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:23:16.640 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:23:16.641 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:23:16.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:23:16.642 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:23:16.643 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:23:16.643 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:23:16.655 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:23:16.655 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.655 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.655 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:23:16.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:23:16.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:23:16.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2023-01-25T12:23:16.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:23:16.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:23:16.658 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:23:16.658 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:23:16.658 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:23:16.658 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:23:16.658 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:23:16.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:23:16.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:23:16.660 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:23:16.660 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:23:16.660 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:23:16.660 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:23:16.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:23:16.661 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:23:16.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:23:16.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:23:16.662 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:23:16.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:23:16.664 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:23:16.665 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:23:16.665 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.665 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.665 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:23:16.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:23:16.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:23:16.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2023-01-25T12:23:16.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:23:16.668 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:23:16.669 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:23:16.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:23:16.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:23:16.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:23:16.670 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:23:16.670 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:23:16.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:23:16.671 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:23:16.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:23:16.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:23:16.671 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:23:16.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:23:16.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:23:16.673 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:23:16.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:23:16.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:23:16.675 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:23:16.675 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.675 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:23:16.676 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:23:16.676 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:23:16.677 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:23:16.677 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:23:16.677 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.677 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.677 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:23:16.679 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:23:16.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:23:16.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:23:16.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:23:16.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2023-01-25T12:23:16.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2023-01-25T12:23:16.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:94: TEST_profile_custom_to_builtin: local OSDS=3 2023-01-25T12:23:16.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: setup td/mclock-config 2023-01-25T12:23:16.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:23:16.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:23:16.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:23:16.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:23:16.682 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:23:16.682 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:23:16.682 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:23:16.682 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:23:16.682 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:23:16.683 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:23:16.684 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:23:16.684 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:23:16.684 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:23:16.685 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:23:16.685 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:23:16.685 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:23:16.685 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:23:16.685 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.686 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:23:16.686 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:23:16.686 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:23:16.686 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:23:16.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:23:16.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:23:16.688 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:23:16.688 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.688 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:23:16.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:23:16.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:23:16.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:23:16.691 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:23:16.691 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:16.691 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:16.691 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:23:16.691 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:23:16.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:23:16.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:23:16.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:23:16.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mon td/mclock-config a --osd_pool_default_size=3 2023-01-25T12:23:16.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/mclock-config 2023-01-25T12:23:16.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-01-25T12:23:16.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-01-25T12:23:16.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-01-25T12:23:16.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/mclock-config/a 2023-01-25T12:23:16.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config --osd_pool_default_size=3 2023-01-25T12:23:17.050 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-01-25T12:23:17.051 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:23:17.051 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:23:17.051 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:23:17.051 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:17.052 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:17.052 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:17.052 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2023-01-25T12:23:17.335 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-01-25T12:23:17.337 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-01-25T12:23:17.337 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:23:17.338 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:23:17.338 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=fsid 2023-01-25T12:23:17.338 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .fsid 2023-01-25T12:23:17.338 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:23:17.338 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:23:17.339 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:23:17.339 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:17.339 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:17.339 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:17.340 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:23:17.340 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:23:17.340 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get fsid 2023-01-25T12:23:17.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-01-25T12:23:17.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:23:17.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:23:17.440 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=mon_host 2023-01-25T12:23:17.440 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .mon_host 2023-01-25T12:23:17.440 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:23:17.440 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:23:17.440 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:23:17.441 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:23:17.441 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:17.441 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:17.441 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:23:17.441 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:23:17.442 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get mon_host 2023-01-25T12:23:17.539 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2023-01-25T12:23:17.540 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/mclock-config 2023-01-25T12:23:17.540 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-01-25T12:23:17.540 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-01-25T12:23:17.540 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-01-25T12:23:17.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/mclock-config/x 2023-01-25T12:23:17.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-01-25T12:23:18.420 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-01-25T12:23:18.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:23:18.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:23:18.421 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:23:18.421 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:18.421 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:18.422 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:18.422 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:23:18.422 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:23:18.447 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:99: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:23:18.449 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:99: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:23:18.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:99: TEST_profile_custom_to_builtin: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:18.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2023-01-25T12:23:18.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:23:18.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:23:18.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-01-25T12:23:18.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:23:18.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/0 2023-01-25T12:23:18.454 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:23:18.455 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:23:18.455 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:23:18.455 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:23:18.455 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2023-01-25T12:23:18.456 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2023-01-25T12:23:18.456 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:23:18.456 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:23:18.456 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:23:18.456 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:23:18.457 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:23:18.457 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:23:18.457 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:23:18.457 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:18.457 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:18.458 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:18.458 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:18.458 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:23:18.458 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:23:18.458 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:23:18.459 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:23:18.459 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:23:18.459 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:23:18.459 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:23:18.459 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:23:18.460 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:23:18.460 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:23:18.460 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/0 2023-01-25T12:23:18.460 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:23:18.461 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=05e59dd4-f858-4aff-8c15-7a666396fc33 2023-01-25T12:23:18.461 INFO:tasks.workunit.client.0.smithi090.stdout:add osd0 05e59dd4-f858-4aff-8c15-7a666396fc33 2023-01-25T12:23:18.461 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd0 05e59dd4-f858-4aff-8c15-7a666396fc33' 2023-01-25T12:23:18.462 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:23:18.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQA2H9FjM1fvGxAADSjElfTehblTVTh2bX7qJw== 2023-01-25T12:23:18.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQA2H9FjM1fvGxAADSjElfTehblTVTh2bX7qJw=="}' 2023-01-25T12:23:18.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 05e59dd4-f858-4aff-8c15-7a666396fc33 -i td/mclock-config/0/new.json 2023-01-25T12:23:18.839 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:23:18.850 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/0/new.json 2023-01-25T12:23:18.851 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQA2H9FjM1fvGxAADSjElfTehblTVTh2bX7qJw== --osd-uuid 05e59dd4-f858-4aff-8c15-7a666396fc33 2023-01-25T12:23:18.877 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.875+0000 7f1135988580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:18.879 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.877+0000 7f1135988580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:18.880 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.878+0000 7f1135988580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:18.880 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.879+0000 7f1135988580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:23:18.881 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.879+0000 7f1135988580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:23:18.881 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.879+0000 7f1135988580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:23:18.963 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:18.961+0000 7f1135988580 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2023-01-25T12:23:24.296 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/0/keyring 2023-01-25T12:23:24.297 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:23:24.297 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd0 key to auth repository 2023-01-25T12:23:24.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd0 key to auth repository 2023-01-25T12:23:24.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:23:24.628 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.0 2023-01-25T12:23:24.629 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.0 2023-01-25T12:23:24.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:23:24.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:23:24.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:23:24.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:23:24.653 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:24.651+0000 7f2fc45bf580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:24.655 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:24.653+0000 7f2fc45bf580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:24.656 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:24.654+0000 7f2fc45bf580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:25.107 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 0 2023-01-25T12:23:25.107 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:23:25.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=0 2023-01-25T12:23:25.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:23:25.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:23:25.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:25.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:23:25.109 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:23:25.109 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:25.110 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:25.179 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:25.178+0000 7f2fc45bf580 -1 Falling back to public interface 2023-01-25T12:23:25.566 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:26.568 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:26.568 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:26.568 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:23:26.569 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:23:26.569 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:26.570 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:27.027 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:27.841 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:27.840+0000 7f2fc45bf580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:23:28.028 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:28.029 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:28.029 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:23:28.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:23:28.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:28.030 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:28.539 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:29.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:29.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:29.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:23:29.542 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:23:29.542 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:29.542 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:29.974 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:30.976 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:30.976 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:30.976 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:23:30.977 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:23:30.977 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:30.977 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:31.511 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:32.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:32.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:32.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:23:32.514 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:23:32.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:32.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:32.954 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:33.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:33.956 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:33.956 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:23:33.956 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:23:33.957 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:33.957 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:34.562 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:35.564 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:35.564 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:35.564 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:23:35.565 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:23:35.565 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:35.566 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:36.014 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:37.016 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:37.016 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:37.016 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:23:37.017 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:23:37.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:37.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:37.541 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:38.542 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:38.543 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:38.543 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:23:38.544 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:23:38.544 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:38.544 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:38.977 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:39.978 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:39.979 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:39.979 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:23:39.980 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:23:39.981 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:39.981 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:41.380 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:42.382 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:42.382 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:42.382 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:23:42.383 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:23:42.383 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:42.383 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:43.824 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:44.825 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:44.826 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:44.826 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:23:44.826 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:23:44.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:44.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:45.263 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:46.264 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:46.264 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:46.265 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:23:46.265 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:23:46.266 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:46.266 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:47.416 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:48.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:48.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:48.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:23:48.419 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:23:48.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:48.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:49.999 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:51.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:51.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:51.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:23:51.002 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:23:51.002 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:51.003 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:52.343 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:53.344 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:53.345 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:53.345 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:23:53.345 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:23:53.346 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:53.346 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:53.984 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:54.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:54.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:54.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:23:54.986 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:23:54.987 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:54.987 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:56.483 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:23:57.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:23:57.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:23:57.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:23:57.486 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:23:57.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:23:57.486 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:23:57.929 INFO:tasks.workunit.client.0.smithi090.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/95930,v1:127.0.0.1:6803/95930] [v2:127.0.0.1:6804/95930,v1:127.0.0.1:6805/95930] exists,up 05e59dd4-f858-4aff-8c15-7a666396fc33 2023-01-25T12:23:57.933 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:23:57.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:23:57.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:23:57.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:99: TEST_profile_custom_to_builtin: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:23:57.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: run_osd td/mclock-config 1 --osd_op_queue=mclock_scheduler 2023-01-25T12:23:57.934 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:23:57.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:23:57.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-01-25T12:23:57.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:23:57.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/1 2023-01-25T12:23:57.935 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:23:57.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:23:57.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:23:57.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:23:57.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/1' 2023-01-25T12:23:57.936 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/1/journal' 2023-01-25T12:23:57.937 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:23:57.937 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:23:57.937 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:23:57.937 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:23:57.937 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:23:57.938 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:23:57.938 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:23:57.938 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:23:57.938 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:23:57.939 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:57.939 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:23:57.939 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:23:57.939 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:23:57.939 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:23:57.940 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:23:57.940 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:23:57.940 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:23:57.940 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:23:57.940 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:23:57.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:23:57.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:23:57.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/1 2023-01-25T12:23:57.941 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:23:57.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040 2023-01-25T12:23:57.942 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd1 142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040' 2023-01-25T12:23:57.942 INFO:tasks.workunit.client.0.smithi090.stdout:add osd1 142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040 2023-01-25T12:23:57.942 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:23:57.949 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQBdH9Fjk06FOBAAer57zZNS9izZqEj74gSFWg== 2023-01-25T12:23:57.950 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQBdH9Fjk06FOBAAer57zZNS9izZqEj74gSFWg=="}' 2023-01-25T12:23:57.950 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040 -i td/mclock-config/1/new.json 2023-01-25T12:23:58.512 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:23:58.523 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/1/new.json 2023-01-25T12:23:58.524 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 1 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/1 --osd-journal=td/mclock-config/1/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQBdH9Fjk06FOBAAer57zZNS9izZqEj74gSFWg== --osd-uuid 142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040 2023-01-25T12:23:58.550 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.549+0000 7f9ea285b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:58.562 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.561+0000 7f9ea285b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:58.562 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.562+0000 7f9ea285b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:23:58.563 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.562+0000 7f9ea285b580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:23:58.563 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.562+0000 7f9ea285b580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:23:58.563 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.562+0000 7f9ea285b580 -1 bluestore(td/mclock-config/1/block) _read_bdev_label failed to open td/mclock-config/1/block: (2) No such file or directory 2023-01-25T12:23:58.654 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:23:58.653+0000 7f9ea285b580 -1 bluestore(td/mclock-config/1) _read_fsid unparsable uuid 2023-01-25T12:24:04.512 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/1/keyring 2023-01-25T12:24:04.512 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:24:04.513 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd1 key to auth repository 2023-01-25T12:24:04.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd1 key to auth repository 2023-01-25T12:24:04.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:24:05.102 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.1 2023-01-25T12:24:05.102 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.1 2023-01-25T12:24:05.103 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 1 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/1 --osd-journal=td/mclock-config/1/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:24:05.103 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:24:05.103 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:24:05.104 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:24:05.126 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:05.125+0000 7fb74f50c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:05.128 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:05.127+0000 7fb74f50c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:05.129 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:05.128+0000 7fb74f50c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:05.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 1 2023-01-25T12:24:05.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:24:05.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=1 2023-01-25T12:24:05.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:24:05.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:24:05.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:05.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:24:05.559 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:24:05.559 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:05.559 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:05.659 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:05.658+0000 7fb74f50c580 -1 Falling back to public interface 2023-01-25T12:24:06.024 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:07.026 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:07.026 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:07.026 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:24:07.027 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:24:07.027 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:07.028 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:07.478 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:08.083 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:08.082+0000 7fb74f50c580 -1 osd.1 0 log_to_monitors true 2023-01-25T12:24:08.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:08.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:08.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:24:08.480 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:24:08.480 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:08.480 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:09.015 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:10.016 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:10.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:10.017 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:24:10.017 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:24:10.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:10.018 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:10.449 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:11.450 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:11.451 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:11.451 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:24:11.452 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:24:11.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:11.453 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:11.982 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:12.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:12.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:12.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:24:12.986 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:24:12.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:12.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:13.430 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:14.431 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:14.431 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:14.432 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:24:14.432 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:24:14.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:14.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:14.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:15.956 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:15.957 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:15.957 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:24:15.957 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:24:15.958 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:15.958 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:16.390 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:17.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:17.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:17.392 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:24:17.392 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:24:17.393 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:17.393 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:17.853 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:18.854 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:18.855 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:18.855 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:24:18.855 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:24:18.856 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:18.856 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:19.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:20.289 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:20.290 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:20.290 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:24:20.290 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:24:20.291 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:20.291 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:21.609 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:22.610 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:22.610 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:22.610 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:24:22.611 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:24:22.611 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:22.612 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:23.754 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:24.756 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:24.756 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:24.756 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:24:24.757 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:24:24.757 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:24.757 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:25.217 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:26.218 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:26.219 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:26.219 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:24:26.219 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:24:26.220 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:26.220 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:27.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:28.175 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:28.176 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:28.176 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:24:28.176 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:24:28.177 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:28.177 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:29.741 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:30.743 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:30.743 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:30.744 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:24:30.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:24:30.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:30.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:31.978 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:32.980 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:32.980 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:32.980 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:24:32.981 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:24:32.981 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:32.982 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:33.641 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:34.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:34.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:34.643 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:24:34.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:24:34.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:34.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:35.707 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:36.708 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:36.709 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:36.709 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:24:36.709 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:24:36.710 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:36.711 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:37.353 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:38.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:38.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:38.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:24:38.356 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:24:38.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:38.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:24:38.793 INFO:tasks.workunit.client.0.smithi090.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/97723,v1:127.0.0.1:6811/97723] [v2:127.0.0.1:6812/97723,v1:127.0.0.1:6813/97723] exists,up 142cbfb4-c31f-4ab8-ae0e-dba2c4e9e040 2023-01-25T12:24:38.797 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:24:38.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:24:38.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:24:38.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:99: TEST_profile_custom_to_builtin: for osd in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:24:38.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: run_osd td/mclock-config 2 --osd_op_queue=mclock_scheduler 2023-01-25T12:24:38.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:24:38.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:24:38.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-01-25T12:24:38.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:24:38.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/2 2023-01-25T12:24:38.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:24:38.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:24:38.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:24:38.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:24:38.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/2' 2023-01-25T12:24:38.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/2/journal' 2023-01-25T12:24:38.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:24:38.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:24:38.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:24:38.801 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:24:38.801 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:24:38.802 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:24:38.802 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:24:38.802 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:24:38.802 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:24:38.802 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:24:38.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:24:38.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:24:38.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:24:38.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:24:38.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:24:38.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:24:38.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:24:38.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:24:38.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:24:38.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:24:38.805 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:24:38.805 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/2 2023-01-25T12:24:38.805 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:24:38.805 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=21a6bdb1-bb87-4431-bc65-20ec210796dd 2023-01-25T12:24:38.805 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd2 21a6bdb1-bb87-4431-bc65-20ec210796dd' 2023-01-25T12:24:38.806 INFO:tasks.workunit.client.0.smithi090.stdout:add osd2 21a6bdb1-bb87-4431-bc65-20ec210796dd 2023-01-25T12:24:38.806 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:24:38.814 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQCGH9FjIVhxMBAAXu/RVblz1pgDFRoHHZapSw== 2023-01-25T12:24:38.814 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQCGH9FjIVhxMBAAXu/RVblz1pgDFRoHHZapSw=="}' 2023-01-25T12:24:38.814 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 21a6bdb1-bb87-4431-bc65-20ec210796dd -i td/mclock-config/2/new.json 2023-01-25T12:24:39.346 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:24:39.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/2/new.json 2023-01-25T12:24:39.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 2 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/2 --osd-journal=td/mclock-config/2/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQCGH9FjIVhxMBAAXu/RVblz1pgDFRoHHZapSw== --osd-uuid 21a6bdb1-bb87-4431-bc65-20ec210796dd 2023-01-25T12:24:39.385 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.383+0000 7fe4b5579580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:39.386 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.385+0000 7fe4b5579580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:39.387 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.386+0000 7fe4b5579580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:39.388 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.386+0000 7fe4b5579580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:24:39.388 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.386+0000 7fe4b5579580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:24:39.389 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.386+0000 7fe4b5579580 -1 bluestore(td/mclock-config/2/block) _read_bdev_label failed to open td/mclock-config/2/block: (2) No such file or directory 2023-01-25T12:24:39.479 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:39.477+0000 7fe4b5579580 -1 bluestore(td/mclock-config/2) _read_fsid unparsable uuid 2023-01-25T12:24:45.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/2/keyring 2023-01-25T12:24:45.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:24:45.662 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd2 key to auth repository 2023-01-25T12:24:45.662 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd2 key to auth repository 2023-01-25T12:24:45.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:24:46.250 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.2 2023-01-25T12:24:46.250 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.2 2023-01-25T12:24:46.251 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 2 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/2 --osd-journal=td/mclock-config/2/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:24:46.251 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:24:46.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:24:46.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:24:46.274 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:46.273+0000 7ff6ab431580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:46.276 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:46.275+0000 7ff6ab431580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:46.277 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:46.276+0000 7ff6ab431580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:24:46.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 2 2023-01-25T12:24:46.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:24:46.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=2 2023-01-25T12:24:46.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:24:46.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:24:46.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:46.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:24:46.680 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:24:46.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:46.681 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:46.810 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:46.809+0000 7ff6ab431580 -1 Falling back to public interface 2023-01-25T12:24:47.127 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:48.129 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:48.129 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:48.130 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:24:48.130 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:24:48.131 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:48.131 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:48.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:49.211 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:24:49.210+0000 7ff6ab431580 -1 osd.2 0 log_to_monitors true 2023-01-25T12:24:49.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:49.606 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:49.606 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:24:49.606 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:24:49.607 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:49.607 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:50.268 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:51.269 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:51.270 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:51.270 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:24:51.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:24:51.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:51.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:51.749 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:52.750 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:52.750 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:52.750 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:24:52.751 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:24:52.751 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:52.752 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:53.324 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:54.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:54.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:54.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:24:54.326 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:24:54.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:54.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:54.810 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:55.811 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:55.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:55.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:24:55.812 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:24:55.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:55.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:56.248 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:57.249 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:57.250 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:57.250 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:24:57.250 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:24:57.251 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:57.251 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:57.684 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:24:58.686 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:24:58.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:24:58.687 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:24:58.688 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:24:58.688 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:24:58.689 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:24:59.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:00.148 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:00.148 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:00.148 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:25:00.149 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:25:00.149 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:00.149 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:00.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:01.631 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:01.632 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:01.632 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:25:01.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:25:01.634 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:01.634 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:02.218 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:03.219 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:03.376 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:03.377 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:25:03.378 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:25:03.378 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:03.378 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:04.196 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:05.198 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:05.198 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:05.198 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:25:05.199 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:25:05.199 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:05.199 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:06.555 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:07.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:07.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:07.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:25:07.558 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:25:07.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:07.558 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:08.076 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:09.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:09.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:09.077 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:25:09.078 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:25:09.078 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:09.079 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:09.602 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:10.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:10.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:10.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:25:10.605 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:25:10.606 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:10.606 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:11.482 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:12.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:12.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:12.484 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:25:12.485 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:25:12.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:12.485 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:14.177 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:15.179 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:15.180 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:15.180 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:25:15.180 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:25:15.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:15.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:16.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:17.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:17.173 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:17.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:25:17.174 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:25:17.174 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:17.175 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:18.460 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:19.461 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:19.462 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:19.463 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:25:19.464 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:25:19.464 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:19.465 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:25:19.953 INFO:tasks.workunit.client.0.smithi090.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/99534,v1:127.0.0.1:6819/99534] [v2:127.0.0.1:6820/99534,v1:127.0.0.1:6821/99534] exists,up 21a6bdb1-bb87-4431-bc65-20ec210796dd 2023-01-25T12:25:19.954 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:25:19.954 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:25:19.954 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:25:19.955 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:105: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:19.955 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:105: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:19.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:105: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:19.956 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2023-01-25T12:25:20.556 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:20.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:20.557 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:105: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:20.557 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: ceph config get osd.1 osd_mclock_profile 2023-01-25T12:25:20.998 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:20.998 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:20.998 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:105: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:20.999 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: ceph config get osd.2 osd_mclock_profile 2023-01-25T12:25:21.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:107: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:21.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:21.493 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:112: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_profile custom 2023-01-25T12:25:22.018 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:115: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:22.018 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:115: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:22.019 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:115: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:22.020 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2023-01-25T12:25:22.448 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: local mclock_profile=custom 2023-01-25T12:25:22.448 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: test custom = custom 2023-01-25T12:25:22.449 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:115: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:22.449 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: ceph config get osd.1 osd_mclock_profile 2023-01-25T12:25:22.874 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: local mclock_profile=custom 2023-01-25T12:25:22.875 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: test custom = custom 2023-01-25T12:25:22.875 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:115: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:22.875 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: ceph config get osd.2 osd_mclock_profile 2023-01-25T12:25:23.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:117: TEST_profile_custom_to_builtin: local mclock_profile=custom 2023-01-25T12:25:23.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: test custom = custom 2023-01-25T12:25:23.325 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:122: TEST_profile_custom_to_builtin: client_res=() 2023-01-25T12:25:23.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:122: TEST_profile_custom_to_builtin: local client_res 2023-01-25T12:25:23.326 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:23.326 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:23.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:23.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: client_res+=($(CEPH_ARGS='' ceph --format=json daemon $(get_asok_path osd.$id) config get osd_mclock_scheduler_client_res | jq .osd_mclock_scheduler_client_res | bc)) 2023-01-25T12:25:23.328 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:23.329 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: get_asok_path osd.0 2023-01-25T12:25:23.329 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:23.329 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:25:23.330 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:23.331 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:23.435 INFO:tasks.workunit.client.0.smithi090.stdout:Original client_res for osd.0 = 17 2023-01-25T12:25:23.436 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:128: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 17' 2023-01-25T12:25:23.437 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:23.437 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: client_res+=($(CEPH_ARGS='' ceph --format=json daemon $(get_asok_path osd.$id) config get osd_mclock_scheduler_client_res | jq .osd_mclock_scheduler_client_res | bc)) 2023-01-25T12:25:23.437 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:23.437 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: get_asok_path osd.1 2023-01-25T12:25:23.437 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:23.438 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-01-25T12:25:23.438 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-01-25T12:25:23.438 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:23.438 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:23.438 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:23.439 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.1.asok 2023-01-25T12:25:23.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:23.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.1.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:23.542 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:128: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.1 = 17' 2023-01-25T12:25:23.543 INFO:tasks.workunit.client.0.smithi090.stdout:Original client_res for osd.1 = 17 2023-01-25T12:25:23.545 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:23.545 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: client_res+=($(CEPH_ARGS='' ceph --format=json daemon $(get_asok_path osd.$id) config get osd_mclock_scheduler_client_res | jq .osd_mclock_scheduler_client_res | bc)) 2023-01-25T12:25:23.545 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:23.546 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:23.546 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: get_asok_path osd.2 2023-01-25T12:25:23.546 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:25:23.546 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:25:23.546 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:23.547 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:23.547 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:23.547 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:25:23.547 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:23.547 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:23.650 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:128: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.2 = 18' 2023-01-25T12:25:23.651 INFO:tasks.workunit.client.0.smithi090.stdout:Original client_res for osd.2 = 18 2023-01-25T12:25:23.651 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: expr 17 + 10 2023-01-25T12:25:23.651 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: local client_res_new=27 2023-01-25T12:25:23.652 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:133: TEST_profile_custom_to_builtin: echo 'client_res_new = 27' 2023-01-25T12:25:23.652 INFO:tasks.workunit.client.0.smithi090.stdout:client_res_new = 27 2023-01-25T12:25:23.653 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:134: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res 27 2023-01-25T12:25:24.150 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:24.152 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:24.152 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:24.153 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2023-01-25T12:25:24.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:24.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:141: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:24.605 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:24.605 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:24.605 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: get_asok_path osd.0 2023-01-25T12:25:24.605 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:25:24.605 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:25:24.606 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:24.606 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:24.606 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:24.606 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:25:24.606 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:24.607 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:24.712 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:24.712 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:146: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:24.713 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:24.713 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: ceph config get osd.1 osd_mclock_scheduler_client_res 2023-01-25T12:25:25.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:25.163 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:141: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:25.164 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:25.165 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:25.165 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: get_asok_path osd.1 2023-01-25T12:25:25.165 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-01-25T12:25:25.165 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-01-25T12:25:25.166 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:25.166 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:25.166 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:25.166 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.1.asok 2023-01-25T12:25:25.166 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:25.167 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.1.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:25.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:25.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:146: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:25.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:25.271 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: ceph config get osd.2 osd_mclock_scheduler_client_res 2023-01-25T12:25:25.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:140: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:25.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:141: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:25.725 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: get_asok_path osd.2 2023-01-25T12:25:25.726 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:25:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:25:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:25.727 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:25.727 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:25.727 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:25:25.727 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:25.727 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:25.832 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:145: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:25.832 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:146: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:25.833 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:155: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_profile high_client_ops 2023-01-25T12:25:26.297 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:157: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:26.298 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:157: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:26.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:157: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:26.299 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2023-01-25T12:25:26.829 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:26.830 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:26.830 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:157: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:26.830 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: ceph config get osd.1 osd_mclock_profile 2023-01-25T12:25:27.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:27.321 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:27.321 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:157: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:27.321 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: ceph config get osd.2 osd_mclock_profile 2023-01-25T12:25:27.805 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:159: TEST_profile_custom_to_builtin: local mclock_profile=high_client_ops 2023-01-25T12:25:27.806 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_client_ops = high_client_ops 2023-01-25T12:25:27.806 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:164: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:27.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:164: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:27.808 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:164: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:27.808 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2023-01-25T12:25:28.239 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:28.239 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:169: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:28.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:28.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:28.241 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: get_asok_path osd.0 2023-01-25T12:25:28.241 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:25:28.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:25:28.242 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:28.242 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:28.242 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:28.242 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:25:28.243 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:28.243 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:28.349 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:28.350 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:174: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:28.350 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:164: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:28.350 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: ceph config get osd.1 osd_mclock_scheduler_client_res 2023-01-25T12:25:28.802 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:28.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:169: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:28.804 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:28.804 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:28.805 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: get_asok_path osd.1 2023-01-25T12:25:28.805 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-01-25T12:25:28.805 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-01-25T12:25:28.805 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:28.805 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:28.806 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:28.806 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.1.asok 2023-01-25T12:25:28.806 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:28.806 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.1.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:28.911 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:28.912 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:174: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:28.912 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:164: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:28.912 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: ceph config get osd.2 osd_mclock_scheduler_client_res 2023-01-25T12:25:29.361 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:168: TEST_profile_custom_to_builtin: local res=27 2023-01-25T12:25:29.362 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:169: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:29.363 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:29.363 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:29.363 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: get_asok_path osd.2 2023-01-25T12:25:29.364 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:25:29.364 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:25:29.365 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:29.365 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:29.365 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:29.365 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:25:29.366 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:29.366 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:29.469 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: res=27 2023-01-25T12:25:29.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:174: TEST_profile_custom_to_builtin: test 27 -eq 27 2023-01-25T12:25:29.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2023-01-25T12:25:29.952 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:181: TEST_profile_custom_to_builtin: expr 3 - 1 2023-01-25T12:25:29.953 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:181: TEST_profile_custom_to_builtin: seq 0 2 2023-01-25T12:25:29.955 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:181: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:29.956 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:29.956 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:29.956 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: get_asok_path osd.0 2023-01-25T12:25:29.956 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:25:29.956 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:25:29.957 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:29.957 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:29.957 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:29.957 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:25:29.957 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:29.958 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:30.062 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: res=17 2023-01-25T12:25:30.063 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:187: TEST_profile_custom_to_builtin: test 17 -eq 17 2023-01-25T12:25:30.063 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:181: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:30.064 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:30.065 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:30.065 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: get_asok_path osd.1 2023-01-25T12:25:30.065 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2023-01-25T12:25:30.065 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2023-01-25T12:25:30.066 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:30.066 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.066 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.066 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.1.asok 2023-01-25T12:25:30.066 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:30.067 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.1.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:30.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: res=17 2023-01-25T12:25:30.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:187: TEST_profile_custom_to_builtin: test 17 -eq 17 2023-01-25T12:25:30.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:181: TEST_profile_custom_to_builtin: for id in $(seq 0 $(expr $OSDS - 1)) 2023-01-25T12:25:30.173 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2023-01-25T12:25:30.174 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc 2023-01-25T12:25:30.174 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: get_asok_path osd.2 2023-01-25T12:25:30.174 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2023-01-25T12:25:30.174 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2023-01-25T12:25:30.174 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:30.175 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.175 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.175 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.2.asok 2023-01-25T12:25:30.175 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: CEPH_ARGS= 2023-01-25T12:25:30.175 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.2.asok config get osd_mclock_scheduler_client_res 2023-01-25T12:25:30.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: res=18 2023-01-25T12:25:30.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:187: TEST_profile_custom_to_builtin: test 18 -eq 18 2023-01-25T12:25:30.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:190: TEST_profile_custom_to_builtin: teardown td/mclock-config 2023-01-25T12:25:30.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:25:30.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:25:30.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:25:30.281 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:25:30.281 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:25:30.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:25:30.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:25:30.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:25:30.387 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:25:30.387 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:25:30.388 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:25:30.388 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:25:30.389 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:25:30.389 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:25:30.389 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:25:30.390 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:25:30.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:25:30.392 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.392 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:25:30.392 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:25:30.393 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:25:30.404 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:25:30.405 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.405 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.406 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:25:30.407 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:25:30.407 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:25:30.408 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2023-01-25T12:25:30.408 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:25:30.408 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:25:30.408 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:25:30.408 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:25:30.409 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:25:30.409 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:25:30.409 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:25:30.409 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:25:30.409 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:25:30.410 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:25:30.412 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:25:30.412 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:25:30.412 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:25:30.413 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:25:30.413 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:25:30.413 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.413 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:25:30.414 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:25:30.414 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.414 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:25:30.417 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:25:30.417 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:25:30.417 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:25:30.417 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.417 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:25:30.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:25:30.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:25:30.418 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:25:30.419 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:25:30.420 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:25:30.420 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:25:30.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:25:30.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:25:30.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:25:30.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:25:30.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:25:30.423 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:25:30.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:25:30.424 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:25:30.424 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:25:30.424 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:25:30.424 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.424 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:25:30.425 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:25:30.425 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.425 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:25:30.425 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:25:30.426 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:25:30.429 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:25:30.429 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.429 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.429 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:25:30.430 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:25:30.430 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:25:30.430 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:25:30.430 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:25:30.431 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.431 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.431 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:25:30.432 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:25:30.432 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:25:30.432 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:25:30.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:25:30.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2023-01-25T12:25:30.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2023-01-25T12:25:30.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:196: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2023-01-25T12:25:30.433 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/mclock-config 2023-01-25T12:25:30.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/mclock-config 2023-01-25T12:25:30.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:25:30.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:25:30.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:25:30.434 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:25:30.435 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:25:30.435 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:25:30.435 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:25:30.435 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:25:30.435 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:25:30.438 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:25:30.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:25:30.438 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:25:30.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:25:30.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:25:30.439 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:25:30.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:25:30.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:25:30.440 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:25:30.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:25:30.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:25:30.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:25:30.440 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:25:30.441 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.441 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.441 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:25:30.441 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:25:30.441 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:25:30.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/mclock-config 2023-01-25T12:25:30.442 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:25:30.442 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.442 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.86354 2023-01-25T12:25:30.444 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:25:30.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2023-01-25T12:25:30.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:25:30.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2023-01-25T12:25:30.444 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:197: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2023-01-25T12:25:30.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/mclock-config 2023-01-25T12:25:30.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-01-25T12:25:30.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-01-25T12:25:30.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-01-25T12:25:30.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/mclock-config/a 2023-01-25T12:25:30.446 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2023-01-25T12:25:30.818 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-01-25T12:25:30.818 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:25:30.819 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:25:30.819 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:25:30.819 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:30.819 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:30.819 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:25:30.820 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2023-01-25T12:25:31.095 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-01-25T12:25:31.097 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-01-25T12:25:31.097 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:25:31.098 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:25:31.098 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=fsid 2023-01-25T12:25:31.098 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .fsid 2023-01-25T12:25:31.098 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:25:31.098 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:25:31.099 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:25:31.099 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:31.099 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:31.099 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:31.099 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:25:31.100 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:25:31.100 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get fsid 2023-01-25T12:25:31.198 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-01-25T12:25:31.198 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:25:31.198 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:25:31.199 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=mon_host 2023-01-25T12:25:31.199 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .mon_host 2023-01-25T12:25:31.199 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:25:31.199 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:25:31.200 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:25:31.200 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:25:31.200 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:31.200 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:31.200 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-mon.a.asok 2023-01-25T12:25:31.201 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:25:31.201 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.86354/ceph-mon.a.asok config get mon_host 2023-01-25T12:25:31.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2023-01-25T12:25:31.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/mclock-config 2023-01-25T12:25:31.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-01-25T12:25:31.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-01-25T12:25:31.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-01-25T12:25:31.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/mclock-config/x 2023-01-25T12:25:31.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-01-25T12:25:32.169 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-01-25T12:25:32.170 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:25:32.170 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:25:32.170 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:25:32.170 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:32.170 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:32.171 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:25:32.172 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:25:32.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:25:32.200 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:200: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2023-01-25T12:25:32.200 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/mclock-config 2023-01-25T12:25:32.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:25:32.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-01-25T12:25:32.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:25:32.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/mclock-config/0 2023-01-25T12:25:32.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2023-01-25T12:25:32.202 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:25:32.202 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:25:32.202 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:25:32.203 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2023-01-25T12:25:32.203 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2023-01-25T12:25:32.204 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:25:32.204 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:25:32.204 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2023-01-25T12:25:32.205 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:25:32.205 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:25:32.205 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:25:32.205 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:25:32.206 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:25:32.206 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:25:32.206 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:25:32.206 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' 2023-01-25T12:25:32.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:25:32.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:25:32.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:25:32.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2023-01-25T12:25:32.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2023-01-25T12:25:32.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:25:32.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:25:32.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:25:32.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:25:32.209 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2023-01-25T12:25:32.209 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/mclock-config/0 2023-01-25T12:25:32.209 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:25:32.212 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=db4f820f-4f5c-437d-914b-8731a732abea 2023-01-25T12:25:32.213 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd0 db4f820f-4f5c-437d-914b-8731a732abea' 2023-01-25T12:25:32.213 INFO:tasks.workunit.client.0.smithi090.stdout:add osd0 db4f820f-4f5c-437d-914b-8731a732abea 2023-01-25T12:25:32.214 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:25:32.218 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQC8H9FjzojrDBAAicNxpgB7BD18Otqo8Kl5CQ== 2023-01-25T12:25:32.218 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQC8H9FjzojrDBAAicNxpgB7BD18Otqo8Kl5CQ=="}' 2023-01-25T12:25:32.218 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new db4f820f-4f5c-437d-914b-8731a732abea -i td/mclock-config/0/new.json 2023-01-25T12:25:32.575 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:25:32.587 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/mclock-config/0/new.json 2023-01-25T12:25:32.587 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler --mkfs --key AQC8H9FjzojrDBAAicNxpgB7BD18Otqo8Kl5CQ== --osd-uuid db4f820f-4f5c-437d-914b-8731a732abea 2023-01-25T12:25:32.613 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.611+0000 7f92df18b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:32.615 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.614+0000 7f92df18b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:32.616 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.614+0000 7f92df18b580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:32.616 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.615+0000 7f92df18b580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:25:32.617 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.615+0000 7f92df18b580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:25:32.617 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.615+0000 7f92df18b580 -1 bluestore(td/mclock-config/0/block) _read_bdev_label failed to open td/mclock-config/0/block: (2) No such file or directory 2023-01-25T12:25:32.691 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:32.689+0000 7f92df18b580 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2023-01-25T12:25:37.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/mclock-config/0/keyring 2023-01-25T12:25:37.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:25:37.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd0 key to auth repository 2023-01-25T12:25:37.986 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd0 key to auth repository 2023-01-25T12:25:37.986 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:25:38.232 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.0 2023-01-25T12:25:38.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.0 2023-01-25T12:25:38.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:25:38.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 0 --fsid=282ee7c1-f362-4e99-897b-3a5ca65bda52 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.86354/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd_op_queue=mclock_scheduler 2023-01-25T12:25:38.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:25:38.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:25:38.257 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:38.255+0000 7fb82a5c6580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:38.268 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:38.266+0000 7fb82a5c6580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:38.269 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:38.267+0000 7fb82a5c6580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:25:38.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 0 2023-01-25T12:25:38.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:25:38.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=0 2023-01-25T12:25:38.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:25:38.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:25:38.727 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:38.727 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:25:38.727 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:25:38.728 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:38.728 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:38.807 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:38.805+0000 7fb82a5c6580 -1 Falling back to public interface 2023-01-25T12:25:39.201 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:40.202 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:40.203 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:40.203 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:25:40.203 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:25:40.204 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:40.204 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:40.634 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:41.258 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:25:41.257+0000 7fb82a5c6580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:25:41.635 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:41.636 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:41.636 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:25:41.637 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:25:41.638 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:41.638 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:42.088 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:43.090 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:43.090 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:43.090 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:25:43.091 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:25:43.091 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:43.092 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:43.520 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:44.522 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:44.522 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:44.522 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:25:44.522 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:25:44.523 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:44.523 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:45.119 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:46.121 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:46.121 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:46.122 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:25:46.122 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:25:46.123 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:46.123 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:46.579 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:47.582 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:47.582 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:47.583 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:25:47.583 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:25:47.583 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:47.584 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:48.114 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:49.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:49.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:49.116 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:25:49.117 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:25:49.117 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:49.117 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:49.569 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:50.570 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:50.570 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:50.571 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:25:50.571 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:25:50.572 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:50.572 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:51.083 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:52.084 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:52.084 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:52.084 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:25:52.085 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:25:52.085 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:52.085 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:52.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:53.517 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:53.517 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:53.517 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:25:53.518 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:25:53.518 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:53.518 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:55.031 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:56.037 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:56.038 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:56.038 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:25:56.038 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:25:56.039 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:56.039 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:56.991 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:25:57.993 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:25:57.993 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:25:57.994 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:25:57.994 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:25:57.995 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:25:57.995 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:25:59.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:00.253 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:00.254 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:00.254 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:26:00.254 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:26:00.255 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:00.255 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:01.787 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:02.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:02.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:02.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:26:02.789 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:26:02.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:02.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:03.842 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:04.844 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:04.844 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:04.845 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:26:04.845 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:26:04.846 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:04.846 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:05.404 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:06.405 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:06.405 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:06.406 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:26:06.406 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:26:06.407 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:06.407 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:08.303 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:09.304 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:09.305 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:09.305 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:26:09.305 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:26:09.306 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:09.306 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:10.142 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:11.143 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:11.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:11.144 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:26:11.145 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:26:11.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:11.145 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:11.600 INFO:tasks.workunit.client.0.smithi090.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/102267,v1:127.0.0.1:6803/102267] [v2:127.0.0.1:6804/102267,v1:127.0.0.1:6805/102267] exists,up db4f820f-4f5c-437d-914b-8731a732abea 2023-01-25T12:26:11.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:26:11.602 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:26:11.602 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:26:11.602 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:26:11.602 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:26:11.603 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:26:11.604 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_recovery_max_active 2023-01-25T12:26:11.707 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2023-01-25T12:26:11.708 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2023-01-25T12:26:11.708 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2023-01-25T12:26:11.710 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:209: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2023-01-25T12:26:12.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: sleep 2 2023-01-25T12:26:14.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:26:14.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:26:14.331 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:26:14.332 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:26:14.332 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:14.332 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:14.332 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:26:14.333 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:26:14.333 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_recovery_max_active 2023-01-25T12:26:14.449 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2023-01-25T12:26:14.450 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:213: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2023-01-25T12:26:14.450 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2023-01-25T12:26:14.929 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2023-01-25T12:26:15.414 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:219: TEST_recovery_limit_adjustment_mclock: sleep 2 2023-01-25T12:26:17.417 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2023-01-25T12:26:17.418 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:26:17.418 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:26:17.418 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:26:17.418 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.418 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:17.419 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.86354/ceph-osd.0.asok 2023-01-25T12:26:17.419 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2023-01-25T12:26:17.419 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.86354/ceph-osd.0.asok config get osd_recovery_max_active 2023-01-25T12:26:17.536 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2023-01-25T12:26:17.536 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:222: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2023-01-25T12:26:17.537 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:224: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2023-01-25T12:26:17.537 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:26:17.537 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:26:17.537 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:26:17.537 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:26:17.538 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:26:17.538 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:26:17.538 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:26:17.538 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:26:17.642 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:26:17.642 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:26:17.643 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:26:17.643 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:26:17.643 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:26:17.644 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:26:17.648 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:26:17.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:26:17.648 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:26:17.649 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.649 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:26:17.649 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:26:17.649 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:26:17.653 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:26:17.653 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.653 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:17.653 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:26:17.655 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:26:17.655 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:26:17.655 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2023-01-25T12:26:17.655 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:26:17.656 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:26:17.656 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:26:17.656 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:26:17.656 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:26:17.656 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:26:17.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:26:17.657 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:26:17.658 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:26:17.658 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:26:17.658 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:26:17.658 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:26:17.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:26:17.659 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:26:17.659 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:26:17.660 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:26:17.661 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:26:17.661 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:26:17.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:26:17.663 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:26:17.664 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:26:17.664 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.664 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:17.664 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:26:17.666 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:26:17.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:26:17.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2244: main: code=0 2023-01-25T12:26:17.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2248: main: teardown td/mclock-config 0 2023-01-25T12:26:17.667 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/mclock-config 2023-01-25T12:26:17.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2023-01-25T12:26:17.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/mclock-config KILL 2023-01-25T12:26:17.668 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:26:17.668 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:26:17.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:26:17.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:26:17.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:26:17.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:26:17.669 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:26:17.669 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:26:17.671 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:26:17.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:26:17.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:26:17.671 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:26:17.671 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:26:17.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:26:17.672 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:26:17.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2023-01-25T12:26:17.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/mclock-config 2023-01-25T12:26:17.674 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:26:17.675 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.675 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.86354 2023-01-25T12:26:17.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.86354 2023-01-25T12:26:17.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:26:17.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:26:17.676 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2249: main: return 0 2023-01-25T12:26:17.676 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-25T12:26:17.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-25T12:26:17.717 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2023-01-25T12:26:17.717 DEBUG:teuthology.orchestra.run.smithi090:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=44e4095da12a8269eb6cfe163cbb0d00b1a6412c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2023-01-25T12:26:17.781 INFO:tasks.workunit.client.0.smithi090.stderr:stty: 'standard input': Inappropriate ioctl for device 2023-01-25T12:26:17.789 INFO:tasks.workunit.client.0.smithi090.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2023-01-25T12:26:17.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2023-01-25T12:26:17.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2023-01-25T12:26:17.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: main: export PYTHONWARNINGS=ignore 2023-01-25T12:26:17.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2238: main: PYTHONWARNINGS=ignore 2023-01-25T12:26:17.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: main: export CEPH_CONF=/dev/null 2023-01-25T12:26:17.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: main: CEPH_CONF=/dev/null 2023-01-25T12:26:17.791 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2240: main: unset CEPH_ARGS 2023-01-25T12:26:17.791 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2242: main: local code 2023-01-25T12:26:17.791 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2243: main: run td/network-ping 2023-01-25T12:26:17.791 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2023-01-25T12:26:17.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2023-01-25T12:26:17.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2023-01-25T12:26:17.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2023-01-25T12:26:17.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2023-01-25T12:26:17.793 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2023-01-25T12:26:17.793 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none ' 2023-01-25T12:26:17.793 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2023-01-25T12:26:17.793 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2023-01-25T12:26:17.794 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2023-01-25T12:26:17.794 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2023-01-25T12:26:17.794 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2023-01-25T12:26:17.794 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2023-01-25T12:26:17.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2023-01-25T12:26:17.798 INFO:tasks.workunit.client.0.smithi090.stderr:TEST_network_ping_test2' 2023-01-25T12:26:17.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2023-01-25T12:26:17.798 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2023-01-25T12:26:17.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/network-ping 2023-01-25T12:26:17.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/network-ping 2023-01-25T12:26:17.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/network-ping 2023-01-25T12:26:17.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:26:17.799 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/network-ping KILL 2023-01-25T12:26:17.800 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:26:17.800 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:26:17.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:26:17.800 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:26:17.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:26:17.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:26:17.801 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:26:17.801 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:26:17.802 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:26:17.802 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:26:17.802 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:26:17.802 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:26:17.802 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:26:17.803 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:26:17.803 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:26:17.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:26:17.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:26:17.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/network-ping 2023-01-25T12:26:17.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:26:17.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:17.808 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.103410 2023-01-25T12:26:17.808 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2023-01-25T12:26:17.808 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2023-01-25T12:26:17.808 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/network-ping 2023-01-25T12:26:17.808 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2023-01-25T12:26:17.809 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:17.809 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:17.809 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.103410 2023-01-25T12:26:17.810 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2023-01-25T12:26:17.810 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2023-01-25T12:26:17.810 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2023-01-25T12:26:17.811 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2023-01-25T12:26:17.811 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2023-01-25T12:26:17.811 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2023-01-25T12:26:17.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2023-01-25T12:26:17.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2023-01-25T12:26:17.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/network-ping 2023-01-25T12:26:17.812 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2023-01-25T12:26:17.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2023-01-25T12:26:17.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2023-01-25T12:26:17.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/network-ping/a 2023-01-25T12:26:17.813 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2023-01-25T12:26:18.204 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2023-01-25T12:26:18.205 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:26:18.205 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:26:18.205 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:26:18.205 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:18.206 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:18.206 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:18.206 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2023-01-25T12:26:18.473 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2023-01-25T12:26:18.475 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2023-01-25T12:26:18.475 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:26:18.476 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:26:18.476 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=fsid 2023-01-25T12:26:18.476 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .fsid 2023-01-25T12:26:18.476 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:26:18.476 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:26:18.477 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:26:18.477 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:26:18.477 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:18.477 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:18.477 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mon.a.asok 2023-01-25T12:26:18.478 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:26:18.478 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.103410/ceph-mon.a.asok config get fsid 2023-01-25T12:26:18.577 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2023-01-25T12:26:18.577 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: local daemon=mon 2023-01-25T12:26:18.578 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1124: get_config: local id=a 2023-01-25T12:26:18.578 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local config=mon_host 2023-01-25T12:26:18.578 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: get_config: jq -r .mon_host 2023-01-25T12:26:18.578 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: get_asok_path mon.a 2023-01-25T12:26:18.578 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2023-01-25T12:26:18.579 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2023-01-25T12:26:18.579 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:26:18.579 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:18.579 INFO:tasks.workunit.client.0.smithi090.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:18.579 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mon.a.asok 2023-01-25T12:26:18.580 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: CEPH_ARGS= 2023-01-25T12:26:18.580 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: ceph --format json daemon /tmp/ceph-asok.103410/ceph-mon.a.asok config get mon_host 2023-01-25T12:26:18.678 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2023-01-25T12:26:18.678 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/network-ping 2023-01-25T12:26:18.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2023-01-25T12:26:18.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2023-01-25T12:26:18.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2023-01-25T12:26:18.679 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/network-ping/x 2023-01-25T12:26:18.680 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2023-01-25T12:26:19.565 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2023-01-25T12:26:19.565 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:26:19.566 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:26:19.566 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:26:19.566 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:19.566 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:19.567 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:19.567 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:26:19.567 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2023-01-25T12:26:19.598 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2023-01-25T12:26:19.599 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/network-ping 2023-01-25T12:26:19.599 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:26:19.599 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2023-01-25T12:26:19.599 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:26:19.600 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/network-ping/0 2023-01-25T12:26:19.600 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2023-01-25T12:26:19.600 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:26:19.600 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:26:19.600 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:26:19.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2023-01-25T12:26:19.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2023-01-25T12:26:19.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:26:19.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:26:19.601 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/network-ping' 2023-01-25T12:26:19.602 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:26:19.602 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:26:19.602 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:26:19.602 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:26:19.602 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:19.603 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:19.603 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:19.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:19.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:26:19.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:26:19.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:26:19.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2023-01-25T12:26:19.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2023-01-25T12:26:19.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:26:19.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+= 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/network-ping/0 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:26:19.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=7a319599-a748-464d-885c-4abbd4a4ad0a 2023-01-25T12:26:19.606 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd0 7a319599-a748-464d-885c-4abbd4a4ad0a' 2023-01-25T12:26:19.606 INFO:tasks.workunit.client.0.smithi090.stdout:add osd0 7a319599-a748-464d-885c-4abbd4a4ad0a 2023-01-25T12:26:19.607 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:26:19.617 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQDrH9FjvGSxJBAA4iUYYsFuwMUrI85hiI5Rbw== 2023-01-25T12:26:19.617 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQDrH9FjvGSxJBAA4iUYYsFuwMUrI85hiI5Rbw=="}' 2023-01-25T12:26:19.617 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 7a319599-a748-464d-885c-4abbd4a4ad0a -i td/network-ping/0/new.json 2023-01-25T12:26:19.977 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:26:19.990 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/network-ping/0/new.json 2023-01-25T12:26:19.991 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 0 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQDrH9FjvGSxJBAA4iUYYsFuwMUrI85hiI5Rbw== --osd-uuid 7a319599-a748-464d-885c-4abbd4a4ad0a 2023-01-25T12:26:20.017 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.015+0000 7ff553f16580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:20.018 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.017+0000 7ff553f16580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:20.019 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.018+0000 7ff553f16580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:20.020 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.018+0000 7ff553f16580 -1 bluestore(td/network-ping/0/block) _read_bdev_label failed to open td/network-ping/0/block: (2) No such file or directory 2023-01-25T12:26:20.020 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.018+0000 7ff553f16580 -1 bluestore(td/network-ping/0/block) _read_bdev_label failed to open td/network-ping/0/block: (2) No such file or directory 2023-01-25T12:26:20.021 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.018+0000 7ff553f16580 -1 bluestore(td/network-ping/0/block) _read_bdev_label failed to open td/network-ping/0/block: (2) No such file or directory 2023-01-25T12:26:20.097 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:20.095+0000 7ff553f16580 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2023-01-25T12:26:25.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/network-ping/0/keyring 2023-01-25T12:26:25.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:26:25.439 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd0 key to auth repository 2023-01-25T12:26:25.439 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd0 key to auth repository 2023-01-25T12:26:25.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:26:25.724 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.0 2023-01-25T12:26:25.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.0 2023-01-25T12:26:25.725 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 0 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:26:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:26:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:26:25.726 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:26:25.750 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:25.749+0000 7f5c0318c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:25.753 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:25.751+0000 7f5c0318c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:25.754 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:25.752+0000 7f5c0318c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:26.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 0 2023-01-25T12:26:26.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:26:26.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=0 2023-01-25T12:26:26.170 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:26:26.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:26:26.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:26.171 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:26:26.171 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:26:26.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:26.172 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:26.274 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:26.272+0000 7f5c0318c580 -1 Falling back to public interface 2023-01-25T12:26:26.695 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:27.697 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:27.697 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:27.697 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:26:27.698 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:26:27.698 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:27.698 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:28.179 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:28.772 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:28.771+0000 7f5c0318c580 -1 osd.0 0 log_to_monitors true 2023-01-25T12:26:29.179 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:29.179 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:26:29.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:26:29.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:29.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:29.772 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:30.774 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:30.774 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:30.774 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:26:30.775 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:26:30.775 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:30.775 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:31.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:32.209 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:32.210 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:32.210 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:26:32.210 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:26:32.211 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:32.211 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:32.856 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:33.858 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:33.859 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:33.859 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:26:33.860 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:26:33.860 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:33.861 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:34.298 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:35.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:35.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:35.301 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:26:35.301 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:26:35.302 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:35.302 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:35.736 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:36.737 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:36.738 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:36.738 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:26:36.738 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:26:36.739 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:36.739 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:37.184 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:38.185 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:38.185 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:38.185 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:26:38.186 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:26:38.187 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:38.188 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:38.734 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:39.735 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:39.735 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:39.735 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:26:39.736 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:26:39.736 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:39.737 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:40.212 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:41.214 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:41.214 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:41.214 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:26:41.215 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:26:41.215 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:41.215 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:42.868 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:43.869 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:43.870 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:43.870 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:26:43.870 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:26:43.871 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:43.871 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:44.924 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:45.926 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:45.926 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:45.926 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:26:45.927 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:26:45.927 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:45.927 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:47.430 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:48.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:48.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:48.434 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:26:48.435 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:26:48.435 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:48.436 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:48.985 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:49.987 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:49.987 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:49.988 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:26:49.988 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:26:49.989 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:49.989 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:51.324 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:52.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:52.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:52.326 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:26:52.327 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:26:52.327 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:52.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:52.840 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:53.842 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:53.842 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:53.842 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:26:53.843 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:26:53.843 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:53.843 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:54.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:55.744 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:55.744 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:55.744 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:26:55.745 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:26:55.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:55.745 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:57.293 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:26:58.294 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:26:58.295 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:26:58.295 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:26:58.296 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:26:58.296 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:26:58.296 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.0 up' 2023-01-25T12:26:58.873 INFO:tasks.workunit.client.0.smithi090.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/104542,v1:127.0.0.1:6803/104542] [v2:127.0.0.1:6804/104542,v1:127.0.0.1:6805/104542] exists,up 7a319599-a748-464d-885c-4abbd4a4ad0a 2023-01-25T12:26:58.877 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:26:58.877 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:26:58.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:26:58.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2023-01-25T12:26:58.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/network-ping 2023-01-25T12:26:58.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:26:58.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2023-01-25T12:26:58.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:26:58.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/network-ping/1 2023-01-25T12:26:58.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2023-01-25T12:26:58.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:26:58.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:26:58.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:26:58.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2023-01-25T12:26:58.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2023-01-25T12:26:58.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:26:58.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:26:58.881 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/network-ping' 2023-01-25T12:26:58.881 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:26:58.881 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:26:58.881 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:26:58.881 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:26:58.882 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:26:58.882 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:26:58.882 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:58.882 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:26:58.882 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:26:58.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:26:58.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:26:58.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2023-01-25T12:26:58.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2023-01-25T12:26:58.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:26:58.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:26:58.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:26:58.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:26:58.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+= 2023-01-25T12:26:58.884 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/network-ping/1 2023-01-25T12:26:58.885 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:26:58.885 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=5de0ef6b-94db-4c10-8d56-6dfdf93c824b 2023-01-25T12:26:58.885 INFO:tasks.workunit.client.0.smithi090.stdout:add osd1 5de0ef6b-94db-4c10-8d56-6dfdf93c824b 2023-01-25T12:26:58.886 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd1 5de0ef6b-94db-4c10-8d56-6dfdf93c824b' 2023-01-25T12:26:58.886 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:26:58.895 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQASINFjsDVLNRAAQIC8qU7xSd/IlSGsvUZFag== 2023-01-25T12:26:58.896 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQASINFjsDVLNRAAQIC8qU7xSd/IlSGsvUZFag=="}' 2023-01-25T12:26:58.896 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 5de0ef6b-94db-4c10-8d56-6dfdf93c824b -i td/network-ping/1/new.json 2023-01-25T12:26:59.468 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:26:59.478 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/network-ping/1/new.json 2023-01-25T12:26:59.479 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 1 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQASINFjsDVLNRAAQIC8qU7xSd/IlSGsvUZFag== --osd-uuid 5de0ef6b-94db-4c10-8d56-6dfdf93c824b 2023-01-25T12:26:59.507 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.506+0000 7f8b2cffc580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:59.509 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.508+0000 7f8b2cffc580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:59.510 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.509+0000 7f8b2cffc580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:26:59.512 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.509+0000 7f8b2cffc580 -1 bluestore(td/network-ping/1/block) _read_bdev_label failed to open td/network-ping/1/block: (2) No such file or directory 2023-01-25T12:26:59.512 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.509+0000 7f8b2cffc580 -1 bluestore(td/network-ping/1/block) _read_bdev_label failed to open td/network-ping/1/block: (2) No such file or directory 2023-01-25T12:26:59.512 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.509+0000 7f8b2cffc580 -1 bluestore(td/network-ping/1/block) _read_bdev_label failed to open td/network-ping/1/block: (2) No such file or directory 2023-01-25T12:26:59.590 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:26:59.589+0000 7f8b2cffc580 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2023-01-25T12:27:05.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/network-ping/1/keyring 2023-01-25T12:27:05.532 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:27:05.533 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd1 key to auth repository 2023-01-25T12:27:05.533 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd1 key to auth repository 2023-01-25T12:27:05.533 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:27:06.156 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.1 2023-01-25T12:27:06.157 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.1 2023-01-25T12:27:06.157 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 1 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:27:06.158 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:27:06.158 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:27:06.158 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:27:06.181 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:06.180+0000 7f23e801e580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:06.184 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:06.182+0000 7f23e801e580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:06.185 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:06.183+0000 7f23e801e580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:06.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 1 2023-01-25T12:27:06.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:27:06.603 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=1 2023-01-25T12:27:06.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:27:06.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:27:06.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:06.604 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:27:06.605 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:27:06.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:06.605 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:06.707 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:06.706+0000 7f23e801e580 -1 Falling back to public interface 2023-01-25T12:27:07.055 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:08.056 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:08.056 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:08.057 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:27:08.057 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:27:08.058 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:08.058 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:08.516 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:09.110 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:09.110+0000 7f23e801e580 -1 osd.1 0 log_to_monitors true 2023-01-25T12:27:09.517 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:09.517 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:09.518 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:27:09.519 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:27:09.519 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:09.519 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:09.990 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:10.991 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:10.991 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:10.992 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:27:10.992 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:27:10.993 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:10.993 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:11.472 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:12.474 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:12.474 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:12.475 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:27:12.475 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:27:12.476 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:12.476 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:13.067 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:14.069 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:14.070 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:14.070 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:27:14.071 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:27:14.071 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:14.071 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:14.526 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:15.527 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:15.528 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:15.528 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:27:15.529 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:27:15.529 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:15.529 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:16.061 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:17.062 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:17.063 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:17.063 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:27:17.063 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:27:17.064 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:17.064 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:17.500 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:18.501 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:18.502 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:18.502 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:27:18.503 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:27:18.503 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:18.503 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:19.019 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:20.020 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:20.020 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:20.021 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:27:20.021 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:27:20.022 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:20.022 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:20.512 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:21.513 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:21.514 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:21.514 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:27:21.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:27:21.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:21.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:22.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:23.903 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:23.904 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:23.904 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:27:23.904 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:27:23.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:23.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:24.848 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:25.849 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:25.850 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:25.850 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:27:25.850 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:27:25.851 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:25.851 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:27.317 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:28.318 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:28.319 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:28.319 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:27:28.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:27:28.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:28.320 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:29.877 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:30.878 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:30.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:30.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:27:30.879 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:27:30.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:30.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:31.364 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:32.365 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:32.366 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:32.366 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:27:32.366 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:27:32.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:32.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:33.718 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:34.720 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:34.720 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:34.720 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:27:34.721 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:27:34.721 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:34.721 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:35.250 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:36.251 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:36.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:36.252 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:27:36.252 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:27:36.253 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:36.253 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:37.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:38.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:38.792 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:38.793 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:27:38.793 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:27:38.793 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:38.794 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.1 up' 2023-01-25T12:27:39.266 INFO:tasks.workunit.client.0.smithi090.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/106335,v1:127.0.0.1:6811/106335] [v2:127.0.0.1:6812/106335,v1:127.0.0.1:6813/106335] exists,up 5de0ef6b-94db-4c10-8d56-6dfdf93c824b 2023-01-25T12:27:39.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:27:39.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:27:39.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:27:39.271 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2023-01-25T12:27:39.272 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/network-ping 2023-01-25T12:27:39.272 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2023-01-25T12:27:39.272 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2023-01-25T12:27:39.272 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2023-01-25T12:27:39.272 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/network-ping/2 2023-01-25T12:27:39.273 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2023-01-25T12:27:39.273 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2023-01-25T12:27:39.273 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2023-01-25T12:27:39.274 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2023-01-25T12:27:39.274 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2023-01-25T12:27:39.274 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2023-01-25T12:27:39.274 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2023-01-25T12:27:39.275 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2023-01-25T12:27:39.275 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/network-ping' 2023-01-25T12:27:39.275 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2023-01-25T12:27:39.275 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2023-01-25T12:27:39.275 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2023-01-25T12:27:39.276 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2023-01-25T12:27:39.276 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:27:39.276 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:27:39.276 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:27:39.276 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' 2023-01-25T12:27:39.277 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2023-01-25T12:27:39.277 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2023-01-25T12:27:39.277 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2023-01-25T12:27:39.277 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2023-01-25T12:27:39.277 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2023-01-25T12:27:39.278 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2023-01-25T12:27:39.278 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2023-01-25T12:27:39.278 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:27:39.278 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' ' 2023-01-25T12:27:39.278 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+= 2023-01-25T12:27:39.279 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: mkdir -p td/network-ping/2 2023-01-25T12:27:39.279 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: uuidgen 2023-01-25T12:27:39.279 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local uuid=23e53b8b-c7b7-4346-8356-53b1984b736a 2023-01-25T12:27:39.279 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: echo 'add osd2 23e53b8b-c7b7-4346-8356-53b1984b736a' 2023-01-25T12:27:39.280 INFO:tasks.workunit.client.0.smithi090.stdout:add osd2 23e53b8b-c7b7-4346-8356-53b1984b736a 2023-01-25T12:27:39.280 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: ceph-authtool --gen-print-key 2023-01-25T12:27:39.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: OSD_SECRET=AQA7INFjpakkERAArLjipnygodYn8hSjDEdTUQ== 2023-01-25T12:27:39.289 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: echo '{"cephx_secret": "AQA7INFjpakkERAArLjipnygodYn8hSjDEdTUQ=="}' 2023-01-25T12:27:39.289 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: ceph osd new 23e53b8b-c7b7-4346-8356-53b1984b736a -i td/network-ping/2/new.json 2023-01-25T12:27:39.847 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:27:39.859 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: rm td/network-ping/2/new.json 2023-01-25T12:27:39.860 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph-osd -i 2 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --mkfs --key AQA7INFjpakkERAArLjipnygodYn8hSjDEdTUQ== --osd-uuid 23e53b8b-c7b7-4346-8356-53b1984b736a 2023-01-25T12:27:39.888 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.886+0000 7f4a5909c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:39.893 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.891+0000 7f4a5909c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:39.894 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.892+0000 7f4a5909c580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:39.894 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.893+0000 7f4a5909c580 -1 bluestore(td/network-ping/2/block) _read_bdev_label failed to open td/network-ping/2/block: (2) No such file or directory 2023-01-25T12:27:39.895 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.893+0000 7f4a5909c580 -1 bluestore(td/network-ping/2/block) _read_bdev_label failed to open td/network-ping/2/block: (2) No such file or directory 2023-01-25T12:27:39.895 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.893+0000 7f4a5909c580 -1 bluestore(td/network-ping/2/block) _read_bdev_label failed to open td/network-ping/2/block: (2) No such file or directory 2023-01-25T12:27:39.941 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:39.940+0000 7f4a5909c580 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2023-01-25T12:27:45.766 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: local key_fn=td/network-ping/2/keyring 2023-01-25T12:27:45.766 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: cat 2023-01-25T12:27:45.767 INFO:tasks.workunit.client.0.smithi090.stdout:adding osd2 key to auth repository 2023-01-25T12:27:45.768 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:674: run_osd: echo adding osd2 key to auth repository 2023-01-25T12:27:45.768 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2023-01-25T12:27:46.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: echo start osd.2 2023-01-25T12:27:46.359 INFO:tasks.workunit.client.0.smithi090.stdout:start osd.2 2023-01-25T12:27:46.360 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: ceph-osd -i 2 --fsid=c56604d7-1beb-4223-b596-4610f159d869 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.103410/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' 2023-01-25T12:27:46.360 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: ceph osd dump --format=json 2023-01-25T12:27:46.360 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: jq '.flags_set[]' 2023-01-25T12:27:46.361 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:680: run_osd: grep -q '"noup"' 2023-01-25T12:27:46.384 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:46.382+0000 7f652d044580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:46.385 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:46.384+0000 7f652d044580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:46.386 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:46.385+0000 7f652d044580 -1 WARNING: all dangerous and experimental features are enabled. 2023-01-25T12:27:46.786 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:683: run_osd: wait_for_osd up 2 2023-01-25T12:27:46.787 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: local state=up 2023-01-25T12:27:46.787 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:977: wait_for_osd: local id=2 2023-01-25T12:27:46.787 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: status=1 2023-01-25T12:27:46.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i=0 )) 2023-01-25T12:27:46.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:46.789 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 0 2023-01-25T12:27:46.790 INFO:tasks.workunit.client.0.smithi090.stdout:0 2023-01-25T12:27:46.790 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:46.791 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:46.922 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:46.921+0000 7f652d044580 -1 Falling back to public interface 2023-01-25T12:27:47.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:48.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:48.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:48.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 1 2023-01-25T12:27:48.286 INFO:tasks.workunit.client.0.smithi090.stdout:1 2023-01-25T12:27:48.286 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:48.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:48.744 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:49.355 INFO:tasks.workunit.client.0.smithi090.stderr:2023-01-25T12:27:49.354+0000 7f652d044580 -1 osd.2 0 log_to_monitors true 2023-01-25T12:27:49.746 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:49.746 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:49.747 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 2 2023-01-25T12:27:49.747 INFO:tasks.workunit.client.0.smithi090.stdout:2 2023-01-25T12:27:49.748 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:49.748 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:50.226 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:51.227 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:51.228 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:51.228 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 3 2023-01-25T12:27:51.228 INFO:tasks.workunit.client.0.smithi090.stdout:3 2023-01-25T12:27:51.229 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:51.229 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:51.712 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:52.713 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:52.714 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:52.714 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 4 2023-01-25T12:27:52.715 INFO:tasks.workunit.client.0.smithi090.stdout:4 2023-01-25T12:27:52.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:52.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:53.207 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:54.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:54.208 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:54.209 INFO:tasks.workunit.client.0.smithi090.stdout:5 2023-01-25T12:27:54.209 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 5 2023-01-25T12:27:54.210 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:54.210 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:54.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:55.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:55.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:55.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 6 2023-01-25T12:27:55.675 INFO:tasks.workunit.client.0.smithi090.stdout:6 2023-01-25T12:27:55.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:55.676 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:56.241 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:57.243 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:57.243 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:57.244 INFO:tasks.workunit.client.0.smithi090.stdout:7 2023-01-25T12:27:57.244 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 7 2023-01-25T12:27:57.244 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:57.245 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:57.727 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:27:58.729 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:27:58.729 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:27:58.729 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 8 2023-01-25T12:27:58.730 INFO:tasks.workunit.client.0.smithi090.stdout:8 2023-01-25T12:27:58.730 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:27:58.730 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:27:59.229 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:00.231 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:00.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:00.233 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 9 2023-01-25T12:28:00.234 INFO:tasks.workunit.client.0.smithi090.stdout:9 2023-01-25T12:28:00.234 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:00.235 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:00.672 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:01.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:01.673 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:01.674 INFO:tasks.workunit.client.0.smithi090.stdout:10 2023-01-25T12:28:01.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 10 2023-01-25T12:28:01.674 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:01.675 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:02.389 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:03.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:03.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:03.391 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 11 2023-01-25T12:28:03.392 INFO:tasks.workunit.client.0.smithi090.stdout:11 2023-01-25T12:28:03.392 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:03.393 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:04.630 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:05.631 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:05.631 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:05.632 INFO:tasks.workunit.client.0.smithi090.stdout:12 2023-01-25T12:28:05.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 12 2023-01-25T12:28:05.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:05.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:07.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:08.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:08.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:08.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 13 2023-01-25T12:28:08.358 INFO:tasks.workunit.client.0.smithi090.stdout:13 2023-01-25T12:28:08.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:08.359 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:08.913 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:09.915 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:09.915 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:09.916 INFO:tasks.workunit.client.0.smithi090.stdout:14 2023-01-25T12:28:09.916 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 14 2023-01-25T12:28:09.916 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:09.917 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:11.179 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:12.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:12.181 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:12.182 INFO:tasks.workunit.client.0.smithi090.stdout:15 2023-01-25T12:28:12.182 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 15 2023-01-25T12:28:12.182 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:12.183 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:12.753 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:13.755 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:13.755 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:13.755 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 16 2023-01-25T12:28:13.755 INFO:tasks.workunit.client.0.smithi090.stdout:16 2023-01-25T12:28:13.756 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:13.756 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:14.918 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:15.920 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:15.920 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:15.920 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 17 2023-01-25T12:28:15.921 INFO:tasks.workunit.client.0.smithi090.stdout:17 2023-01-25T12:28:15.921 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:15.921 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:16.355 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:17.356 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:17.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:17.357 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 18 2023-01-25T12:28:17.357 INFO:tasks.workunit.client.0.smithi090.stdout:18 2023-01-25T12:28:17.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:17.358 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:18.691 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: sleep 1 2023-01-25T12:28:19.692 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i++ )) 2023-01-25T12:28:19.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:980: wait_for_osd: (( i < 300 )) 2023-01-25T12:28:19.693 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: echo 19 2023-01-25T12:28:19.693 INFO:tasks.workunit.client.0.smithi090.stdout:19 2023-01-25T12:28:19.694 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: ceph osd dump 2023-01-25T12:28:19.694 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: grep 'osd.2 up' 2023-01-25T12:28:20.145 INFO:tasks.workunit.client.0.smithi090.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/108120,v1:127.0.0.1:6819/108120] [v2:127.0.0.1:6820/108120,v1:127.0.0.1:6821/108120] exists,up 23e53b8b-c7b7-4346-8356-53b1984b736a 2023-01-25T12:28:20.146 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: status=0 2023-01-25T12:28:20.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:986: wait_for_osd: break 2023-01-25T12:28:20.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:989: wait_for_osd: return 0 2023-01-25T12:28:20.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2023-01-25T12:28:25.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2023-01-25T12:28:25.147 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2023-01-25T12:28:26.181 INFO:tasks.workunit.client.0.smithi090.stderr:pool 'foo' already exists 2023-01-25T12:28:26.191 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2023-01-25T12:28:27.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2023-01-25T12:28:27.337 INFO:tasks.workunit.client.0.smithi090.stdout:hints = 1 2023-01-25T12:28:27.337 INFO:tasks.workunit.client.0.smithi090.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2023-01-25T12:28:27.337 INFO:tasks.workunit.client.0.smithi090.stdout:Object prefix: benchmark_data_smithi090_109131 2023-01-25T12:28:27.338 INFO:tasks.workunit.client.0.smithi090.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2023-01-25T12:28:27.338 INFO:tasks.workunit.client.0.smithi090.stdout: 0 0 0 0 0 0 - 0 2023-01-25T12:28:28.337 INFO:tasks.workunit.client.0.smithi090.stdout: 1 16 64 48 0.18748 0.1875 0.256963 0.302108 2023-01-25T12:28:29.337 INFO:tasks.workunit.client.0.smithi090.stdout: 2 16 123 107 0.208958 0.230469 0.226374 0.283225 2023-01-25T12:28:30.337 INFO:tasks.workunit.client.0.smithi090.stdout: 3 16 171 155 0.201795 0.1875 0.292426 0.299563 2023-01-25T12:28:31.337 INFO:tasks.workunit.client.0.smithi090.stdout: 4 16 224 208 0.2031 0.207031 0.345746 0.296213 2023-01-25T12:28:32.337 INFO:tasks.workunit.client.0.smithi090.stdout: 5 16 272 256 0.199974 0.1875 0.39649 0.299226 2023-01-25T12:28:33.338 INFO:tasks.workunit.client.0.smithi090.stdout: 6 16 320 304 0.19789 0.1875 0.268475 0.305376 2023-01-25T12:28:34.339 INFO:tasks.workunit.client.0.smithi090.stdout: 7 16 368 352 0.196403 0.1875 0.35928 0.312313 2023-01-25T12:28:35.338 INFO:tasks.workunit.client.0.smithi090.stdout: 8 16 416 400 0.195286 0.1875 0.371861 0.314742 2023-01-25T12:28:36.338 INFO:tasks.workunit.client.0.smithi090.stdout: 9 16 459 443 0.192248 0.167969 0.257573 0.316851 2023-01-25T12:28:37.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16 512 496 0.193723 0.207031 0.238783 0.319686 2023-01-25T12:28:38.338 INFO:tasks.workunit.client.0.smithi090.stdout:Total time run: 10.3452 2023-01-25T12:28:38.339 INFO:tasks.workunit.client.0.smithi090.stdout:Total writes made: 512 2023-01-25T12:28:38.339 INFO:tasks.workunit.client.0.smithi090.stdout:Write size: 4096 2023-01-25T12:28:38.339 INFO:tasks.workunit.client.0.smithi090.stdout:Object size: 4096 2023-01-25T12:28:38.339 INFO:tasks.workunit.client.0.smithi090.stdout:Bandwidth (MB/sec): 0.193327 2023-01-25T12:28:38.340 INFO:tasks.workunit.client.0.smithi090.stdout:Stddev Bandwidth: 0.0169971 2023-01-25T12:28:38.340 INFO:tasks.workunit.client.0.smithi090.stdout:Max bandwidth (MB/sec): 0.230469 2023-01-25T12:28:38.340 INFO:tasks.workunit.client.0.smithi090.stdout:Min bandwidth (MB/sec): 0.167969 2023-01-25T12:28:38.340 INFO:tasks.workunit.client.0.smithi090.stdout:Average IOPS: 49 2023-01-25T12:28:38.340 INFO:tasks.workunit.client.0.smithi090.stdout:Stddev IOPS: 4.35125 2023-01-25T12:28:38.341 INFO:tasks.workunit.client.0.smithi090.stdout:Max IOPS: 59 2023-01-25T12:28:38.341 INFO:tasks.workunit.client.0.smithi090.stdout:Min IOPS: 43 2023-01-25T12:28:38.341 INFO:tasks.workunit.client.0.smithi090.stdout:Average Latency(s): 0.31848 2023-01-25T12:28:38.341 INFO:tasks.workunit.client.0.smithi090.stdout:Stddev Latency(s): 0.071641 2023-01-25T12:28:38.341 INFO:tasks.workunit.client.0.smithi090.stdout:Max latency(s): 0.478136 2023-01-25T12:28:38.342 INFO:tasks.workunit.client.0.smithi090.stdout:Min latency(s): 0.222345 2023-01-25T12:28:38.440 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2023-01-25T12:28:48.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2023-01-25T12:28:48.515 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: flush_pg_stats: local timeout=300 2023-01-25T12:28:48.516 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ceph osd ls 2023-01-25T12:28:48.892 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ids='0 2023-01-25T12:28:48.893 INFO:tasks.workunit.client.0.smithi090.stderr:1 2023-01-25T12:28:48.893 INFO:tasks.workunit.client.0.smithi090.stderr:2' 2023-01-25T12:28:48.893 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2130: flush_pg_stats: seqs= 2023-01-25T12:28:48.894 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:28:48.894 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-01-25T12:28:49.055 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=21474836514 2023-01-25T12:28:49.055 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 21474836514 2023-01-25T12:28:49.055 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836514' 2023-01-25T12:28:49.056 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:28:49.056 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-01-25T12:28:49.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=42949672988 2023-01-25T12:28:49.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 42949672988 2023-01-25T12:28:49.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836514 1-42949672988' 2023-01-25T12:28:49.193 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:28:49.194 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-01-25T12:28:49.330 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=64424509462 2023-01-25T12:28:49.330 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 64424509462 2023-01-25T12:28:49.330 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836514 1-42949672988 2-64424509462' 2023-01-25T12:28:49.331 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:28:49.331 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 0-21474836514 2023-01-25T12:28:49.332 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:28:49.333 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=0 2023-01-25T12:28:49.333 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 0-21474836514 2023-01-25T12:28:49.334 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:28:49.334 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=21474836514 2023-01-25T12:28:49.334 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2023-01-25T12:28:49.335 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.0 seq 21474836514 2023-01-25T12:28:49.335 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 0 2023-01-25T12:28:49.785 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 21474836515 -lt 21474836514 2023-01-25T12:28:49.786 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:28:49.786 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 1-42949672988 2023-01-25T12:28:49.786 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:28:49.787 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=1 2023-01-25T12:28:49.787 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 1-42949672988 2023-01-25T12:28:49.788 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:28:49.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=42949672988 2023-01-25T12:28:49.788 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.1 seq 42949672988' 2023-01-25T12:28:49.789 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.1 seq 42949672988 2023-01-25T12:28:49.789 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 1 2023-01-25T12:28:50.280 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 42949672988 -lt 42949672988 2023-01-25T12:28:50.281 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:28:50.281 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 2-64424509462 2023-01-25T12:28:50.282 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:28:50.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=2 2023-01-25T12:28:50.282 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 2-64424509462 2023-01-25T12:28:50.283 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:28:50.283 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=64424509462 2023-01-25T12:28:50.284 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.2 seq 64424509462' 2023-01-25T12:28:50.284 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.2 seq 64424509462 2023-01-25T12:28:50.284 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 2 2023-01-25T12:28:50.765 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 64424509462 -lt 64424509462 2023-01-25T12:28:50.765 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:28:50.766 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:28:50.767 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:28:50.767 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:28:50.767 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:28:50.767 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:28:50.768 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:28:50.768 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:28:50.768 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:28:50.768 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 2023-01-25T12:28:50.866 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:28:50.866 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:28:50.867 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:28:50.867 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:28:50.876 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:50.876 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:28:50.907 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:28:50.907 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:50.908 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2023-01-25T12:28:50.938 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:28:50.939 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:28:50.940 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:28:50.940 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:28:50.940 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:28:50.941 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:28:50.941 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:28:50.941 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:28:50.941 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:28:50.942 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:28:50.942 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 2023-01-25T12:28:51.028 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:28:51.028 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:28:51.029 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:28:51.029 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:28:51.037 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.037 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:28:51.069 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:28:51.069 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.070 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2023-01-25T12:28:51.101 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:28:51.102 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:28:51.102 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:28:51.102 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:28:51.103 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:28:51.103 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:28:51.104 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:28:51.104 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:28:51.104 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:28:51.104 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:28:51.105 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 0 2023-01-25T12:28:51.200 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:28:51.201 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:28:51.201 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:28:51.201 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.201 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:50 2023", 2023-01-25T12:28:51.202 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.202 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.202 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.202 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.202 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.203 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.273, 2023-01-25T12:28:51.203 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.317, 2023-01-25T12:28:51.203 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.335 2023-01-25T12:28:51.203 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.203 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.204 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.389, 2023-01-25T12:28:51.204 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.426, 2023-01-25T12:28:51.204 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.463 2023-01-25T12:28:51.204 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.204 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.205 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.389, 2023-01-25T12:28:51.205 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.426, 2023-01-25T12:28:51.205 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.463 2023-01-25T12:28:51.205 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.205 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.225 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:50 2023", 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.206 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.260, 2023-01-25T12:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.313, 2023-01-25T12:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.331 2023-01-25T12:28:51.208 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.208 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.208 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.362, 2023-01-25T12:28:51.208 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.400, 2023-01-25T12:28:51.208 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:28:51.209 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.209 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.209 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.362, 2023-01-25T12:28:51.209 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.400, 2023-01-25T12:28:51.209 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.194 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:49 2023", 2023-01-25T12:28:51.210 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.211 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.211 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.211 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.211 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.211 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.290, 2023-01-25T12:28:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.301, 2023-01-25T12:28:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.285 2023-01-25T12:28:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.342, 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.423, 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.342, 2023-01-25T12:28:51.213 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.423, 2023-01-25T12:28:51.214 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:28:51.214 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.214 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.203 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:49 2023", 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.215 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.216 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.216 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.216 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.249, 2023-01-25T12:28:51.216 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.300, 2023-01-25T12:28:51.216 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.292 2023-01-25T12:28:51.217 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.217 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.217 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.328, 2023-01-25T12:28:51.217 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.405, 2023-01-25T12:28:51.217 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.442 2023-01-25T12:28:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.328, 2023-01-25T12:28:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.405, 2023-01-25T12:28:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.442 2023-01-25T12:28:51.219 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.219 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.209 2023-01-25T12:28:51.219 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:28:51.219 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:28:51.219 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:28:51.220 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.220 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:28:51.240 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2023-01-25T12:28:51.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.241 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2023-01-25T12:28:51.273 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:28:51.273 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:28:51.273 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:28:51.273 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:28:51.274 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:28:51.274 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:28:51.274 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:28:51.274 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:28:51.275 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:28:51.275 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:28:51.275 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 0 2023-01-25T12:28:51.363 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:28:51.364 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:28:51.364 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:28:51.364 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.364 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.365 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.365 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:28:51.365 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.365 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.365 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.366 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.288, 2023-01-25T12:28:51.366 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.337, 2023-01-25T12:28:51.366 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.364 2023-01-25T12:28:51.366 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.366 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.253, 2023-01-25T12:28:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.253, 2023-01-25T12:28:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.253 2023-01-25T12:28:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.368 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.344, 2023-01-25T12:28:51.368 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.491, 2023-01-25T12:28:51.368 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.491 2023-01-25T12:28:51.368 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.368 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.248 2023-01-25T12:28:51.369 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.369 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.369 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:46 2023", 2023-01-25T12:28:51.369 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.369 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:28:51.370 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.370 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.370 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.370 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.334, 2023-01-25T12:28:51.370 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.349, 2023-01-25T12:28:51.371 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.361 2023-01-25T12:28:51.371 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.371 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.371 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.269, 2023-01-25T12:28:51.371 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.245, 2023-01-25T12:28:51.372 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.245 2023-01-25T12:28:51.372 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.372 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.372 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.398, 2023-01-25T12:28:51.373 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.515, 2023-01-25T12:28:51.373 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.515 2023-01-25T12:28:51.373 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.285 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:46 2023", 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.374 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:28:51.375 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.375 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.375 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.375 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.301, 2023-01-25T12:28:51.375 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.353, 2023-01-25T12:28:51.376 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.349 2023-01-25T12:28:51.376 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.376 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.376 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.243, 2023-01-25T12:28:51.376 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.224, 2023-01-25T12:28:51.377 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.224 2023-01-25T12:28:51.377 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.377 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.377 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.370, 2023-01-25T12:28:51.377 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.777, 2023-01-25T12:28:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.777 2023-01-25T12:28:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.306 2023-01-25T12:28:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.379 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:45 2023", 2023-01-25T12:28:51.379 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.379 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.379 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.379 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.380 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.380 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.316, 2023-01-25T12:28:51.380 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.332, 2023-01-25T12:28:51.380 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.340 2023-01-25T12:28:51.380 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.256, 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.193, 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.193 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.381 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.372, 2023-01-25T12:28:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.463, 2023-01-25T12:28:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.463 2023-01-25T12:28:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.248 2023-01-25T12:28:51.383 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.383 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.383 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:45 2023", 2023-01-25T12:28:51.383 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.383 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.384 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:28:51.384 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.384 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.384 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.322, 2023-01-25T12:28:51.385 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.329, 2023-01-25T12:28:51.385 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.337 2023-01-25T12:28:51.385 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.385 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.385 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.264, 2023-01-25T12:28:51.386 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.183, 2023-01-25T12:28:51.386 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.183 2023-01-25T12:28:51.386 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.386 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.386 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.388, 2023-01-25T12:28:51.387 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.486, 2023-01-25T12:28:51.387 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:28:51.387 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.387 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.230 2023-01-25T12:28:51.387 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:28:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:28:51.389 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.389 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.389 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.264, 2023-01-25T12:28:51.389 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.309, 2023-01-25T12:28:51.389 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.325 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.228, 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.218, 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.218 2023-01-25T12:28:51.390 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.391 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.391 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.357, 2023-01-25T12:28:51.391 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.441, 2023-01-25T12:28:51.391 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.441 2023-01-25T12:28:51.391 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.392 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.284 2023-01-25T12:28:51.392 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.392 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.392 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:45 2023", 2023-01-25T12:28:51.392 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.393 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:28:51.393 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:28:51.393 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.393 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.393 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.276, 2023-01-25T12:28:51.394 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.324, 2023-01-25T12:28:51.394 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.307 2023-01-25T12:28:51.394 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.394 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.394 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.230, 2023-01-25T12:28:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.223, 2023-01-25T12:28:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.198 2023-01-25T12:28:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.353, 2023-01-25T12:28:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.482, 2023-01-25T12:28:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.482 2023-01-25T12:28:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.318 2023-01-25T12:28:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.397 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.397 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.397 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.397 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:28:51.397 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.398 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.398 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.398 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.279, 2023-01-25T12:28:51.398 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.323, 2023-01-25T12:28:51.398 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.323 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.244, 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.221, 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.221 2023-01-25T12:28:51.399 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.400 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.400 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.382, 2023-01-25T12:28:51.400 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.455, 2023-01-25T12:28:51.400 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.455 2023-01-25T12:28:51.401 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.401 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.261 2023-01-25T12:28:51.401 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.401 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.401 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.402 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.402 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.402 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.402 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:28:51.402 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.320, 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.322, 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.296 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.403 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.239, 2023-01-25T12:28:51.404 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.171, 2023-01-25T12:28:51.404 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.171 2023-01-25T12:28:51.404 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.404 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.404 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.363, 2023-01-25T12:28:51.405 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.442, 2023-01-25T12:28:51.405 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.442 2023-01-25T12:28:51.405 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.405 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.239 2023-01-25T12:28:51.405 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:45 2023", 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:28:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.407 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.407 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.292, 2023-01-25T12:28:51.407 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.318, 2023-01-25T12:28:51.407 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.300 2023-01-25T12:28:51.407 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.408 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.408 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.231, 2023-01-25T12:28:51.408 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.197, 2023-01-25T12:28:51.408 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.197 2023-01-25T12:28:51.408 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.326, 2023-01-25T12:28:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.476, 2023-01-25T12:28:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.476 2023-01-25T12:28:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.274 2023-01-25T12:28:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.411 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:28:51.411 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:28:51.411 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.411 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.411 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.266, 2023-01-25T12:28:51.412 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.310, 2023-01-25T12:28:51.412 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.314 2023-01-25T12:28:51.412 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.412 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.412 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.214, 2023-01-25T12:28:51.413 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.209, 2023-01-25T12:28:51.413 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.209 2023-01-25T12:28:51.413 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.413 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.413 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.314, 2023-01-25T12:28:51.414 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.433, 2023-01-25T12:28:51.414 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.433 2023-01-25T12:28:51.414 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.414 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.277 2023-01-25T12:28:51.415 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.415 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:28:51.415 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:28:44 2023", 2023-01-25T12:28:51.415 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:28:51.415 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:28:51.416 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:28:51.416 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:28:51.416 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:28:51.416 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.309, 2023-01-25T12:28:51.416 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.312, 2023-01-25T12:28:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.287 2023-01-25T12:28:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:28:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.220, 2023-01-25T12:28:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.163, 2023-01-25T12:28:51.418 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.163 2023-01-25T12:28:51.418 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.418 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:28:51.418 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.387, 2023-01-25T12:28:51.418 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.432, 2023-01-25T12:28:51.419 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:28:51.419 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:28:51.419 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.282 2023-01-25T12:28:51.419 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:28:51.419 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:28:51.420 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:28:51.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:28:51.421 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2023-01-25T12:28:51.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:28:51.421 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2023-01-25T12:28:51.436 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:28:51.436 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2023-01-25T12:29:11.437 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2023-01-25T12:29:11.438 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: flush_pg_stats: local timeout=300 2023-01-25T12:29:11.438 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ceph osd ls 2023-01-25T12:29:11.946 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ids='0 2023-01-25T12:29:11.947 INFO:tasks.workunit.client.0.smithi090.stderr:1 2023-01-25T12:29:11.947 INFO:tasks.workunit.client.0.smithi090.stderr:2' 2023-01-25T12:29:11.947 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2130: flush_pg_stats: seqs= 2023-01-25T12:29:11.947 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:29:11.948 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-01-25T12:29:12.091 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=21474836521 2023-01-25T12:29:12.091 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 21474836521 2023-01-25T12:29:12.091 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836521' 2023-01-25T12:29:12.091 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:29:12.092 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-01-25T12:29:12.230 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=42949672995 2023-01-25T12:29:12.230 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 42949672995 2023-01-25T12:29:12.231 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836521 1-42949672995' 2023-01-25T12:29:12.231 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:29:12.231 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-01-25T12:29:12.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=64424509469 2023-01-25T12:29:12.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 64424509469 2023-01-25T12:29:12.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836521 1-42949672995 2-64424509469' 2023-01-25T12:29:12.367 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:29:12.368 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 0-21474836521 2023-01-25T12:29:12.368 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:29:12.369 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=0 2023-01-25T12:29:12.369 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 0-21474836521 2023-01-25T12:29:12.370 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:29:12.370 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=21474836521 2023-01-25T12:29:12.370 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2023-01-25T12:29:12.370 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.0 seq 21474836521 2023-01-25T12:29:12.371 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 0 2023-01-25T12:29:12.824 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 21474836519 -lt 21474836521 2023-01-25T12:29:12.825 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: flush_pg_stats: sleep 1 2023-01-25T12:29:13.825 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: flush_pg_stats: '[' 300 -eq 0 ']' 2023-01-25T12:29:13.826 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 0 2023-01-25T12:29:14.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 21474836521 -lt 21474836521 2023-01-25T12:29:14.285 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:29:14.286 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 1-42949672995 2023-01-25T12:29:14.286 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:29:14.287 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=1 2023-01-25T12:29:14.287 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 1-42949672995 2023-01-25T12:29:14.288 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:29:14.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=42949672995 2023-01-25T12:29:14.288 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.1 seq 42949672995' 2023-01-25T12:29:14.289 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.1 seq 42949672995 2023-01-25T12:29:14.289 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 1 2023-01-25T12:29:14.879 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 42949672995 -lt 42949672995 2023-01-25T12:29:14.880 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:29:14.880 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 2-64424509469 2023-01-25T12:29:14.881 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:29:14.881 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=2 2023-01-25T12:29:14.882 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 2-64424509469 2023-01-25T12:29:14.882 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:29:14.882 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=64424509469 2023-01-25T12:29:14.883 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.2 seq 64424509469' 2023-01-25T12:29:14.883 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.2 seq 64424509469 2023-01-25T12:29:14.883 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 2 2023-01-25T12:29:15.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 64424509469 -lt 64424509469 2023-01-25T12:29:15.328 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:29:15.329 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:29:15.330 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:29:15.330 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:29:15.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:29:15.330 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:29:15.331 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:29:15.331 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:29:15.331 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:29:15.331 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 2023-01-25T12:29:15.429 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:29:15.429 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:29:15.429 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:29:15.429 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:29:15.438 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.438 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:29:15.470 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:29:15.471 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.472 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2023-01-25T12:29:15.502 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:29:15.502 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:29:15.503 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:29:15.503 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:29:15.504 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:29:15.504 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:29:15.504 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:29:15.504 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:29:15.505 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:29:15.505 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:29:15.505 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 2023-01-25T12:29:15.592 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:29:15.593 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:29:15.593 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:29:15.593 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:29:15.602 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.602 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:29:15.633 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:29:15.634 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.634 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2023-01-25T12:29:15.665 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:29:15.665 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:29:15.667 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:29:15.667 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:29:15.667 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:29:15.667 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:29:15.668 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:29:15.668 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:29:15.668 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:29:15.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:29:15.668 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 0 2023-01-25T12:29:15.765 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:29:15.766 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:29:15.766 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:29:15.766 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.766 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.766 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.767 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.767 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.767 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.767 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.768 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.315, 2023-01-25T12:29:15.768 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.330, 2023-01-25T12:29:15.768 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.335 2023-01-25T12:29:15.768 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.768 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.769 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.449, 2023-01-25T12:29:15.769 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.469, 2023-01-25T12:29:15.769 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.469 2023-01-25T12:29:15.769 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.770 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.770 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.449, 2023-01-25T12:29:15.770 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.469, 2023-01-25T12:29:15.770 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.469 2023-01-25T12:29:15.770 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.771 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.229 2023-01-25T12:29:15.771 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.771 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.771 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.771 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.772 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.773 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.773 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.773 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.773 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.312, 2023-01-25T12:29:15.773 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.314, 2023-01-25T12:29:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.328 2023-01-25T12:29:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.458, 2023-01-25T12:29:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.458, 2023-01-25T12:29:15.775 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:29:15.775 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.775 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.775 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.458, 2023-01-25T12:29:15.775 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.458, 2023-01-25T12:29:15.776 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:29:15.776 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.777 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.221 2023-01-25T12:29:15.777 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.777 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.777 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.777 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.778 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.778 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.778 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.778 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.779 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.248, 2023-01-25T12:29:15.779 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.306, 2023-01-25T12:29:15.779 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.299 2023-01-25T12:29:15.779 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.779 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.780 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.301, 2023-01-25T12:29:15.780 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.450, 2023-01-25T12:29:15.780 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.450 2023-01-25T12:29:15.780 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.780 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.301, 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.450, 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.450 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.176 2023-01-25T12:29:15.781 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.782 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.782 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.782 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.782 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.782 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.783 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.783 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.783 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.243, 2023-01-25T12:29:15.783 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.296, 2023-01-25T12:29:15.784 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.292 2023-01-25T12:29:15.784 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.784 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.784 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.293, 2023-01-25T12:29:15.784 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.430, 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.293, 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.430, 2023-01-25T12:29:15.785 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:29:15.786 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.786 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.195 2023-01-25T12:29:15.786 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:29:15.786 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:29:15.786 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:29:15.787 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.787 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:29:15.806 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2023-01-25T12:29:15.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.807 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2023-01-25T12:29:15.838 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:29:15.838 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:29:15.840 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:29:15.840 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:29:15.840 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:29:15.840 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:29:15.840 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:29:15.841 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:29:15.841 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:29:15.841 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:29:15.841 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 0 2023-01-25T12:29:15.928 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:29:15.929 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:29:15.929 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:29:15.929 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.929 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:11 2023", 2023-01-25T12:29:15.929 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.930 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:29:15.930 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.930 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.930 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.930 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.326, 2023-01-25T12:29:15.931 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.341, 2023-01-25T12:29:15.931 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.352 2023-01-25T12:29:15.931 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.931 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.932 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.223, 2023-01-25T12:29:15.932 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.223, 2023-01-25T12:29:15.932 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.223 2023-01-25T12:29:15.932 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.932 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.933 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.415, 2023-01-25T12:29:15.933 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.490, 2023-01-25T12:29:15.933 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.515 2023-01-25T12:29:15.933 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.933 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.274 2023-01-25T12:29:15.934 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.934 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.934 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:11 2023", 2023-01-25T12:29:15.934 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.329, 2023-01-25T12:29:15.935 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.328, 2023-01-25T12:29:15.936 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.342 2023-01-25T12:29:15.936 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.936 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.936 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.251, 2023-01-25T12:29:15.937 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.245, 2023-01-25T12:29:15.937 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.224 2023-01-25T12:29:15.937 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.937 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.937 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.404, 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.469, 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.777 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.226 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.938 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.939 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.939 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.939 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:29:15.939 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.939 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.940 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.940 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.312, 2023-01-25T12:29:15.940 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.324, 2023-01-25T12:29:15.940 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.341 2023-01-25T12:29:15.940 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.221, 2023-01-25T12:29:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.221, 2023-01-25T12:29:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.221 2023-01-25T12:29:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.942 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.942 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.391, 2023-01-25T12:29:15.942 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.481, 2023-01-25T12:29:15.942 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.491 2023-01-25T12:29:15.943 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.943 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.282 2023-01-25T12:29:15.943 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.943 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.943 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.944 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.944 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.944 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.944 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.944 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.315, 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.330, 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.335 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.945 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.256, 2023-01-25T12:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.219, 2023-01-25T12:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.193 2023-01-25T12:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.449, 2023-01-25T12:29:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.469, 2023-01-25T12:29:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.469 2023-01-25T12:29:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.327 2023-01-25T12:29:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.948 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.948 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.948 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.948 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:29:15.948 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:29:15.949 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.949 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.949 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.332, 2023-01-25T12:29:15.949 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.333, 2023-01-25T12:29:15.949 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.313 2023-01-25T12:29:15.950 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.950 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.950 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.249, 2023-01-25T12:29:15.950 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.248, 2023-01-25T12:29:15.950 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.198 2023-01-25T12:29:15.951 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.951 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.951 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.456, 2023-01-25T12:29:15.951 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.456, 2023-01-25T12:29:15.951 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.482 2023-01-25T12:29:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.243 2023-01-25T12:29:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.953 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.953 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.953 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:29:15.953 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.953 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.954 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.312, 2023-01-25T12:29:15.954 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.314, 2023-01-25T12:29:15.954 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.328 2023-01-25T12:29:15.955 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.955 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.955 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.250, 2023-01-25T12:29:15.955 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.194, 2023-01-25T12:29:15.955 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.183 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.458, 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.458, 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.486 2023-01-25T12:29:15.956 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.957 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.305 2023-01-25T12:29:15.957 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.957 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.957 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:09 2023", 2023-01-25T12:29:15.957 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.958 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.958 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.958 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.958 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.959 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.328, 2023-01-25T12:29:15.959 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.307, 2023-01-25T12:29:15.959 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.299 2023-01-25T12:29:15.959 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.959 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.267, 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.195, 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.171 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.960 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.421, 2023-01-25T12:29:15.961 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.450, 2023-01-25T12:29:15.961 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.450 2023-01-25T12:29:15.961 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.961 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.280 2023-01-25T12:29:15.962 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.962 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.962 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.962 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.297, 2023-01-25T12:29:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.304, 2023-01-25T12:29:15.964 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.316 2023-01-25T12:29:15.964 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.964 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.964 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.212, 2023-01-25T12:29:15.964 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.212, 2023-01-25T12:29:15.965 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.212 2023-01-25T12:29:15.965 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.965 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.965 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.380, 2023-01-25T12:29:15.966 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.439, 2023-01-25T12:29:15.966 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.455 2023-01-25T12:29:15.966 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.966 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.252 2023-01-25T12:29:15.966 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.967 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.967 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:09 2023", 2023-01-25T12:29:15.967 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.967 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:29:15.967 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:29:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.311, 2023-01-25T12:29:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.305, 2023-01-25T12:29:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.292 2023-01-25T12:29:15.969 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.969 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.969 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.238, 2023-01-25T12:29:15.969 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.203, 2023-01-25T12:29:15.969 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.163 2023-01-25T12:29:15.970 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.970 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.970 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.410, 2023-01-25T12:29:15.970 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.430, 2023-01-25T12:29:15.970 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.432 2023-01-25T12:29:15.971 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.971 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.293 2023-01-25T12:29:15.971 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.971 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.971 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.972 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.972 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:29:15.972 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:29:15.972 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:29:15.972 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.288, 2023-01-25T12:29:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.296, 2023-01-25T12:29:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.310 2023-01-25T12:29:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.974 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.204, 2023-01-25T12:29:15.974 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.204, 2023-01-25T12:29:15.974 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.204 2023-01-25T12:29:15.974 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.975 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.975 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.400, 2023-01-25T12:29:15.975 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.457, 2023-01-25T12:29:15.975 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.457 2023-01-25T12:29:15.975 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.976 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.269 2023-01-25T12:29:15.976 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.976 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.976 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:10 2023", 2023-01-25T12:29:15.977 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.977 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:29:15.977 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:29:15.977 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.977 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.307, 2023-01-25T12:29:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.304, 2023-01-25T12:29:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.299 2023-01-25T12:29:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.979 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.211, 2023-01-25T12:29:15.979 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.206, 2023-01-25T12:29:15.979 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.197 2023-01-25T12:29:15.979 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.979 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.980 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.380, 2023-01-25T12:29:15.980 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.422, 2023-01-25T12:29:15.980 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.476 2023-01-25T12:29:15.980 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.980 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.253 2023-01-25T12:29:15.981 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.981 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:29:15.981 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:29:14 2023", 2023-01-25T12:29:15.981 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:29:15.982 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:29:15.982 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:29:15.982 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:29:15.982 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:29:15.982 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.281, 2023-01-25T12:29:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.287, 2023-01-25T12:29:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.303 2023-01-25T12:29:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:29:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.237, 2023-01-25T12:29:15.984 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.226, 2023-01-25T12:29:15.984 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.209 2023-01-25T12:29:15.984 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.984 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:29:15.984 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.338, 2023-01-25T12:29:15.985 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.431, 2023-01-25T12:29:15.985 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.433 2023-01-25T12:29:15.985 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:29:15.985 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.237 2023-01-25T12:29:15.985 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:29:15.986 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:29:15.986 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:29:15.987 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.987 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:29:15.987 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2023-01-25T12:29:15.987 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:29:15.988 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2023-01-25T12:29:16.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:29:16.001 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2023-01-25T12:30:06.002 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2023-01-25T12:30:06.002 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2127: flush_pg_stats: local timeout=300 2023-01-25T12:30:06.003 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ceph osd ls 2023-01-25T12:30:06.456 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: flush_pg_stats: ids='0 2023-01-25T12:30:06.456 INFO:tasks.workunit.client.0.smithi090.stderr:1 2023-01-25T12:30:06.457 INFO:tasks.workunit.client.0.smithi090.stderr:2' 2023-01-25T12:30:06.457 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2130: flush_pg_stats: seqs= 2023-01-25T12:30:06.457 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:30:06.457 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2023-01-25T12:30:06.628 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=21474836534 2023-01-25T12:30:06.628 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 21474836534 2023-01-25T12:30:06.628 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836534' 2023-01-25T12:30:06.629 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:30:06.629 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2023-01-25T12:30:06.765 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=42949673008 2023-01-25T12:30:06.766 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 42949673008 2023-01-25T12:30:06.766 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836534 1-42949673008' 2023-01-25T12:30:06.766 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: flush_pg_stats: for osd in $ids 2023-01-25T12:30:06.766 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2023-01-25T12:30:06.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: flush_pg_stats: seq=64424509482 2023-01-25T12:30:06.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: flush_pg_stats: test -z 64424509482 2023-01-25T12:30:06.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: flush_pg_stats: seqs=' 0-21474836534 1-42949673008 2-64424509482' 2023-01-25T12:30:06.902 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:30:06.903 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 0-21474836534 2023-01-25T12:30:06.903 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:30:06.903 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=0 2023-01-25T12:30:06.904 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 0-21474836534 2023-01-25T12:30:06.905 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:30:06.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=21474836534 2023-01-25T12:30:06.905 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2023-01-25T12:30:06.906 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.0 seq 21474836534 2023-01-25T12:30:06.906 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 0 2023-01-25T12:30:07.339 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 21474836532 -lt 21474836534 2023-01-25T12:30:07.340 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: flush_pg_stats: sleep 1 2023-01-25T12:30:08.341 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2146: flush_pg_stats: '[' 300 -eq 0 ']' 2023-01-25T12:30:08.341 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 0 2023-01-25T12:30:08.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 21474836534 -lt 21474836534 2023-01-25T12:30:08.804 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:30:08.805 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 1-42949673008 2023-01-25T12:30:08.805 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:30:08.806 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=1 2023-01-25T12:30:08.806 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 1-42949673008 2023-01-25T12:30:08.806 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:30:08.807 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=42949673008 2023-01-25T12:30:08.807 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.1 seq 42949673008' 2023-01-25T12:30:08.807 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.1 seq 42949673008 2023-01-25T12:30:08.808 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 1 2023-01-25T12:30:09.297 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 42949673008 -lt 42949673008 2023-01-25T12:30:09.297 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2140: flush_pg_stats: for s in $seqs 2023-01-25T12:30:09.298 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: echo 2-64424509482 2023-01-25T12:30:09.298 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: cut -d - -f 1 2023-01-25T12:30:09.299 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2141: flush_pg_stats: osd=2 2023-01-25T12:30:09.299 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: echo 2-64424509482 2023-01-25T12:30:09.299 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: cut -d - -f 2 2023-01-25T12:30:09.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: flush_pg_stats: seq=64424509482 2023-01-25T12:30:09.300 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: flush_pg_stats: echo 'waiting osd.2 seq 64424509482' 2023-01-25T12:30:09.301 INFO:tasks.workunit.client.0.smithi090.stdout:waiting osd.2 seq 64424509482 2023-01-25T12:30:09.301 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: ceph osd last-stat-seq 2 2023-01-25T12:30:09.767 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: flush_pg_stats: test 64424509482 -lt 64424509482 2023-01-25T12:30:09.767 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:09.768 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:30:09.768 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:30:09.769 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:30:09.769 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:09.769 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:09.769 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:09.769 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:30:09.770 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:09.770 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 2023-01-25T12:30:09.868 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:09.869 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:30:09.869 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:30:09.869 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:09.878 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:09.878 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:30:09.909 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:30:09.910 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:09.910 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:09.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:30:09.941 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:09.942 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:30:09.943 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:30:09.943 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:30:09.943 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:09.943 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:09.943 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:09.944 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:30:09.944 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:09.944 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 2023-01-25T12:30:10.032 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:10.033 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 1000, 2023-01-25T12:30:10.033 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:30:10.033 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:10.042 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.042 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:30:10.073 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:30:10.074 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.074 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:10.104 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2023-01-25T12:30:10.106 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:10.106 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:30:10.106 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:30:10.107 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:30:10.107 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:10.107 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:10.107 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:10.107 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:30:10.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:10.108 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 0 2023-01-25T12:30:10.206 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:10.206 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:30:10.206 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:30:10.206 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.207 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.208 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.311, 2023-01-25T12:30:10.208 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.331, 2023-01-25T12:30:10.208 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.327 2023-01-25T12:30:10.208 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.389, 2023-01-25T12:30:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.451, 2023-01-25T12:30:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.478 2023-01-25T12:30:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.389, 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.451, 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.478 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.210 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.273 2023-01-25T12:30:10.211 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.211 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.211 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.211 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.211 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.212 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.212 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.212 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.212 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.295, 2023-01-25T12:30:10.212 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.319, 2023-01-25T12:30:10.213 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.312 2023-01-25T12:30:10.213 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.213 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.213 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.366, 2023-01-25T12:30:10.213 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.455, 2023-01-25T12:30:10.214 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.458 2023-01-25T12:30:10.214 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.214 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.214 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.366, 2023-01-25T12:30:10.214 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.455, 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.458 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.248 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:09 2023", 2023-01-25T12:30:10.216 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.216 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.216 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.216 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.216 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.217 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.268, 2023-01-25T12:30:10.217 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.307, 2023-01-25T12:30:10.217 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.307 2023-01-25T12:30:10.217 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.217 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.218 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.308, 2023-01-25T12:30:10.218 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.466, 2023-01-25T12:30:10.219 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.466 2023-01-25T12:30:10.219 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.219 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.220 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.308, 2023-01-25T12:30:10.220 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.466, 2023-01-25T12:30:10.220 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.466 2023-01-25T12:30:10.221 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.221 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.308 2023-01-25T12:30:10.221 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.221 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.221 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:09 2023", 2023-01-25T12:30:10.222 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.222 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.222 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.222 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.222 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.223 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.244, 2023-01-25T12:30:10.223 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.288, 2023-01-25T12:30:10.223 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.292 2023-01-25T12:30:10.223 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.223 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.224 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.273, 2023-01-25T12:30:10.224 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.479, 2023-01-25T12:30:10.224 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.479 2023-01-25T12:30:10.224 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.224 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.273, 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.479, 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.479 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.237 2023-01-25T12:30:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:30:10.226 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:30:10.226 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:10.227 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.227 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:30:10.247 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2023-01-25T12:30:10.247 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.247 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:10.279 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:30:10.279 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:10.280 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:30:10.281 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:30:10.281 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:30:10.281 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:10.281 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:10.281 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:10.282 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:30:10.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:10.282 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 0 2023-01-25T12:30:10.369 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:10.369 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 0, 2023-01-25T12:30:10.369 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [ 2023-01-25T12:30:10.369 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.370 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.370 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.370 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:30:10.370 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.370 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.296, 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.345, 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.329 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.371 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.372 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.234, 2023-01-25T12:30:10.372 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.234, 2023-01-25T12:30:10.372 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.182 2023-01-25T12:30:10.372 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.372 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.373 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.372, 2023-01-25T12:30:10.373 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.471, 2023-01-25T12:30:10.373 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.481 2023-01-25T12:30:10.373 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.373 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.206 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.375 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.375 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.375 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.345, 2023-01-25T12:30:10.375 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.315, 2023-01-25T12:30:10.375 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.309 2023-01-25T12:30:10.376 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.376 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.376 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.287, 2023-01-25T12:30:10.376 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.228, 2023-01-25T12:30:10.376 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.176 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.407, 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.466, 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.466 2023-01-25T12:30:10.377 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.378 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.263 2023-01-25T12:30:10.378 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.378 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.378 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:06 2023", 2023-01-25T12:30:10.378 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:30:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.302, 2023-01-25T12:30:10.380 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.316, 2023-01-25T12:30:10.380 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.331 2023-01-25T12:30:10.380 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.380 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.380 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.291, 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.270, 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.223 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.314, 2023-01-25T12:30:10.381 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.462, 2023-01-25T12:30:10.382 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.503 2023-01-25T12:30:10.382 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.382 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.292 2023-01-25T12:30:10.382 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.382 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.383 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.311, 2023-01-25T12:30:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.331, 2023-01-25T12:30:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.327 2023-01-25T12:30:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.265, 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.251, 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.228 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.385 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.389, 2023-01-25T12:30:10.386 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.451, 2023-01-25T12:30:10.386 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.478 2023-01-25T12:30:10.386 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.386 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.278 2023-01-25T12:30:10.386 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.387 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.387 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.387 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.387 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:30:10.387 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.279, 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.326, 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.308 2023-01-25T12:30:10.388 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.389 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.389 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.249, 2023-01-25T12:30:10.389 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.249, 2023-01-25T12:30:10.389 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.204 2023-01-25T12:30:10.389 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.341, 2023-01-25T12:30:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.447, 2023-01-25T12:30:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.457 2023-01-25T12:30:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.196 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.391 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:30:10.392 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.392 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.392 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.392 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.283, 2023-01-25T12:30:10.392 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.324, 2023-01-25T12:30:10.393 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.314 2023-01-25T12:30:10.393 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.393 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.393 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.226, 2023-01-25T12:30:10.393 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.226, 2023-01-25T12:30:10.394 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.212 2023-01-25T12:30:10.394 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.394 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.394 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.357, 2023-01-25T12:30:10.394 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.431, 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.470 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.227 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.396 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.396 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.396 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.396 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.396 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.397 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.295, 2023-01-25T12:30:10.397 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.319, 2023-01-25T12:30:10.397 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.312 2023-01-25T12:30:10.397 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.397 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.398 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.247, 2023-01-25T12:30:10.398 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.244, 2023-01-25T12:30:10.398 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.209 2023-01-25T12:30:10.398 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.398 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.399 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.366, 2023-01-25T12:30:10.399 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.455, 2023-01-25T12:30:10.399 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.458 2023-01-25T12:30:10.399 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.399 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.244 2023-01-25T12:30:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:06 2023", 2023-01-25T12:30:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 2, 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.258, 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.298, 2023-01-25T12:30:10.401 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.315 2023-01-25T12:30:10.402 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.402 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.402 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.234, 2023-01-25T12:30:10.402 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.234, 2023-01-25T12:30:10.402 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.226 2023-01-25T12:30:10.403 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.403 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.403 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.285, 2023-01-25T12:30:10.403 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.449, 2023-01-25T12:30:10.403 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.472 2023-01-25T12:30:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.240 2023-01-25T12:30:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "front", 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.262, 2023-01-25T12:30:10.406 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.285, 2023-01-25T12:30:10.406 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.311 2023-01-25T12:30:10.406 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.406 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.406 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.253, 2023-01-25T12:30:10.407 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.231, 2023-01-25T12:30:10.407 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.224 2023-01-25T12:30:10.407 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.407 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.407 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.273, 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.427, 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.465 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.246 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.408 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.409 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.409 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.409 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 2, 2023-01-25T12:30:10.409 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:30:10.409 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.410 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.410 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.266, 2023-01-25T12:30:10.410 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.310, 2023-01-25T12:30:10.410 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.295 2023-01-25T12:30:10.410 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.218, 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.218, 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.210 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.412 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.330, 2023-01-25T12:30:10.412 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.480, 2023-01-25T12:30:10.412 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.480 2023-01-25T12:30:10.412 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.412 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.244 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:04 2023", 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 0, 2023-01-25T12:30:10.413 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 1, 2023-01-25T12:30:10.414 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.414 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.414 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.307, 2023-01-25T12:30:10.414 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.292, 2023-01-25T12:30:10.414 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.294 2023-01-25T12:30:10.415 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.415 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.415 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.236, 2023-01-25T12:30:10.415 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.199, 2023-01-25T12:30:10.415 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.195 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.357, 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.479, 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.479 2023-01-25T12:30:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.417 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.253 2023-01-25T12:30:10.417 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.417 INFO:tasks.workunit.client.0.smithi090.stdout: { 2023-01-25T12:30:10.417 INFO:tasks.workunit.client.0.smithi090.stdout: "last update": "Wed Jan 25 12:30:05 2023", 2023-01-25T12:30:10.417 INFO:tasks.workunit.client.0.smithi090.stdout: "stale": false, 2023-01-25T12:30:10.418 INFO:tasks.workunit.client.0.smithi090.stdout: "from osd": 1, 2023-01-25T12:30:10.418 INFO:tasks.workunit.client.0.smithi090.stdout: "to osd": 0, 2023-01-25T12:30:10.418 INFO:tasks.workunit.client.0.smithi090.stdout: "interface": "back", 2023-01-25T12:30:10.418 INFO:tasks.workunit.client.0.smithi090.stdout: "average": { 2023-01-25T12:30:10.418 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.269, 2023-01-25T12:30:10.419 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.273, 2023-01-25T12:30:10.419 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.292 2023-01-25T12:30:10.419 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.419 INFO:tasks.workunit.client.0.smithi090.stdout: "min": { 2023-01-25T12:30:10.419 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.235, 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.217, 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.206 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: "max": { 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: "1min": 0.316, 2023-01-25T12:30:10.420 INFO:tasks.workunit.client.0.smithi090.stdout: "5min": 0.413, 2023-01-25T12:30:10.421 INFO:tasks.workunit.client.0.smithi090.stdout: "15min": 0.440 2023-01-25T12:30:10.421 INFO:tasks.workunit.client.0.smithi090.stdout: }, 2023-01-25T12:30:10.421 INFO:tasks.workunit.client.0.smithi090.stdout: "last": 0.264 2023-01-25T12:30:10.421 INFO:tasks.workunit.client.0.smithi090.stdout: } 2023-01-25T12:30:10.421 INFO:tasks.workunit.client.0.smithi090.stdout: ] 2023-01-25T12:30:10.422 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:10.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2023-01-25T12:30:10.423 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2023-01-25T12:30:10.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.423 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:10.442 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2023-01-25T12:30:10.443 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:10.444 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2023-01-25T12:30:10.444 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mgr.x 2023-01-25T12:30:10.444 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mgr.x ']' 2023-01-25T12:30:10.444 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:10.445 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:10.445 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:10.445 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-mgr.x.asok 2023-01-25T12:30:10.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:10.445 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-mgr.x.asok dump_osd_network 99 2023-01-25T12:30:10.533 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:10.533 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 99, 2023-01-25T12:30:10.533 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:30:10.534 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:10.542 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.542 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:10.573 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2023-01-25T12:30:10.573 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2023-01-25T12:30:10.574 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2023-01-25T12:30:10.575 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2023-01-25T12:30:10.575 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2023-01-25T12:30:10.575 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2023-01-25T12:30:10.575 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:10.576 INFO:tasks.workunit.client.0.smithi090.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-25T12:30:10.576 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.103410/ceph-osd.0.asok 2023-01-25T12:30:10.576 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2023-01-25T12:30:10.576 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.103410/ceph-osd.0.asok dump_osd_network 98 2023-01-25T12:30:10.673 INFO:tasks.workunit.client.0.smithi090.stdout:{ 2023-01-25T12:30:10.673 INFO:tasks.workunit.client.0.smithi090.stdout: "threshold": 98, 2023-01-25T12:30:10.673 INFO:tasks.workunit.client.0.smithi090.stdout: "entries": [] 2023-01-25T12:30:10.673 INFO:tasks.workunit.client.0.smithi090.stdout:} 2023-01-25T12:30:10.682 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2023-01-25T12:30:10.682 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2023-01-25T12:30:10.714 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2023-01-25T12:30:10.714 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2023-01-25T12:30:10.715 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2023-01-25T12:30:10.715 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/network-ping 2023-01-25T12:30:10.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2023-01-25T12:30:10.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/network-ping KILL 2023-01-25T12:30:10.716 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2023-01-25T12:30:10.716 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2023-01-25T12:30:10.716 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2023-01-25T12:30:10.717 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2023-01-25T12:30:10.717 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2023-01-25T12:30:10.823 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2023-01-25T12:30:10.823 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2023-01-25T12:30:10.824 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2023-01-25T12:30:10.824 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2023-01-25T12:30:10.825 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2023-01-25T12:30:10.825 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2023-01-25T12:30:10.825 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2023-01-25T12:30:10.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:30:10.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2023-01-25T12:30:10.827 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2023-01-25T12:30:10.828 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T12:30:10.828 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2023-01-25T12:30:10.829 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2023-01-25T12:30:10.829 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/network-ping 2023-01-25T12:30:10.846 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2023-01-25T12:30:10.846 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2023-01-25T12:30:10.846 INFO:tasks.workunit.client.0.smithi090.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.103410 2023-01-